An experience in the implementation of the flipped classroom instructional model in the computer science degree

In recent years, education has undergone a profound transformation process, having gone from relying only on the traditional lecture to making full use of digital formats. This gradual process, accelerated by the COVID‐19 pandemic in the last years, has triggered innovative changes in the educational process. Within this context, many lecturers have adopted the flipped classroom instructional model, aiming to improve the motivation and involvement of the students. In this model, students must acquire certain theoretical knowledge at home, and the classes, with the help of the lecturer, are used for the more practical part. This article presents the results obtained when the flipped classroom model was implemented in the computer science degree. Specifically, during 2020–2021 and 2021–2022 courses, this instructional model was assessed in 15 subjects. Results show that the flipped classroom instructional model can be stated that it has improved the students' perception of the learning experience, students' dedication and engagement has been improved, students' perception of their understanding of the subject has improved. The faculty considers that the teaching experience has improved and is in favor of continuing with the experience in future courses.

New generation students make extensive use of technology to consume information through multiple devices (e.g., smartwatches, smartphones, tablets, and computers).This habit of consuming technology continuously contrasts with the different realities found for learning process at the university and places an increased demand on the use of technology.Given this scenario, the teaching community is well-known for the opportunities of the use of technology to improve communication, collaboration, and motivation in the learning process.However, it is also aware of the challenge of maintaining the technological skills of the teaching staff, as well as the contradiction that exists among students between the ability to be continuously connected to devices and maintaining concentration for learning.Consequently, the education community is developing a progressive model of incorporating technology into the learning process, balancing the challenges with the benefits.
Following the conclusions reached by neuroscience experts, it has been demonstrated that the motivation and involvement of the student is a determining factor for success in the learning process. 4Based on this theory, numerous innovative pedagogical initiatives have been proposed to increase student participation, mainly through active methodologies and models. 5,6Among the different proposals, it is worth highlighting the flipped classroom instructional model.In the first phase, students work on the contents of a lesson before attending class.This content, such as videos or written material, is developed by lecturers and is made available on technological platforms. 7In a second phase, students participate in additional activities in the classroom to deepen the knowledge acquired under the guidance of the lecturer.By extracting the introductory lecture from the classes to a previous activity, classroom time may be employed for developing lecture concepts by practical exercises, laboratory or active methodologies.
In March 2020, with the emergence of the COVID-19 pandemic, lecturers around the world had no choice but to change their methodologies and rely on information and communication technologies to deliver distance learning.Once the first phase of the pandemic was over, at the beginning of the next academic year (2020/2021), some universities returned to partially face-to-face classes due to space limitations in classrooms and social distancing.In this context, universities adopted the technique known as blended learning, which combines face-to-face and distance learning.This reduction in face-to-face classes has highlighted the need to optimize face-to-face class time.In this sense, innovative teaching methodologies applied to increase student engagement and motivation can also be used to partially address the absence of face-to-face classes.][11] This article presents the work carried out to adopt the flipped classroom instructional model in the first three courses of the university's computer science degree.The reception of this experience has been quantitatively and qualitatively measured, answering the following research questions: • RQ1: Does the flipped classroom instructional model improve the students' perception of the learning experience?
• RQ2: Does the flipped classroom instructional model improve students' dedication/engagement?
• RQ3: Does the flipped classroom instructional model improve students' perception of the understanding of the subject matter?
• RQ4: Has the flipped classroom experience been better than last year?
• RQ5: Is it feasible to implement a flipped classroom experience at a collective level?
• RQ6: Does the flipped classroom approach model improve the teaching experience?
• RQ7: Is the effort to prepare the flipped classroom sessions greater the second year?
The remainder of this document is structured as follows: Section 2 presents related works on the flipped classroom instructional model.Section 3 presents the case study of this work, including the motivation and implementation of the initiative, the description of the phases carried out when adapting the flipped classroom model as well as the evaluation mechanisms employed.Section 4 describes the results obtained in the experience.Finally, Section 6 presents the conclusions and future lines of action.

RELATED WORK
There has been an increase in the implementation of the flipped classroom instructional model in the last decade.Nowadays, students are familiarized with this approach, and they are motivated in the subjects, which implement the method.Flipped classroom instructional model has been implemented in the secondary school but also in the university.Most flipped classroom experiences involve one subject and one lecturer.For example, Mason et al. 12 recorded some videos that students must watch before each session.Afterward, in the classroom, students must figure out all exercises and activities collectively or individually.Meanwhile, the lecturers can help students to resolve their questions.Students completed their projects with hardware and software provided by the university, but they did not check if they had the knowledge by a questionnaire.This experience was carried out with 20 students, and the objective of those responsible for the mechanical degree is to create future workers with an outstanding problem-solving skills.In Gannod's 13 experience, students were required to watch between 3 and 6 h of lectures per week.In class, laboratories require a computer equipped classroom to complete the proposed tasks applying the knowledge acquired in the videos.After class, students were provided with more time to complete the class activities.This experience was implemented for 40 software engineering students.Jovanovic 14 created videos with multiple-choice questions.These questions covered the concepts discussed in the video, and they could answer a question, have the answer evaluated, and if it was incorrect, they could request to either see the solution or try again.This experience is the only one that added documents instead of videos before the class sessions.Students were required to read some documents and answer questions.In class, the students had a sequence of activities to complete related with the video or document information.If an exercise was correctly solved, the student's score was increased, and the exercise was removed from the sequence.Alternatively, a new exercise was randomly selected, and the current problem remained in the sequence.This method was introduced as an incentive for students to prepare for the lecture.Ng 15 created videos for students that should learn from the videos set and read the teaching materials if appropriate.Students attempted a pre-test in class prior to watching the online videos on their own to help them understand the expectations.In fact, there were also some hyperlinks included in the designated websites where students could further enhance their knowledge.During class, students were required to apply their self-learned knowledge by editing a photograph that they could use together in their group.Jong 16 developed the flipped classroom experience for the "teaching and learning with technology" subject.This is mandatory for students of different degrees before starting the university.Once the videos have been watched, Kahoot has been used to find out the level of knowledge of the subject on the part of the students.After this, the different subject activities were carried out.There are more contributions involving one lecturer in one subject.Gren 17 accomplished an experiment with 50 software engineering students with flipped classroom.The first step for the session was to watch videos around 10-20 min long.The first activity in class was to answer a questionnaire in order to obtain the students' knowledge level about the topic before the class activity.The class sessions consisted in 5 to 10 min discussion in pairs, a discussion with the whole class, or in groups or pairs, 5 min of administrative information on labs and lab reports.A group discussion about an online video is done if students consider necessary to increment information from the lecturer.Chiquito et al. 18 implemented a flipped classroom experience with two groups: 97 students used the flipped classroom model and 98 followed the traditional approach.Flipped classroom students were obliged to watch videos on YouTube, and also they must answer a questionnaire to know their knowledge before the class session.The link activity between previous and class session consisted of five short questions, which were developed on Moodle, and opened for students before the class.Using this linked activity, the professor had information about the students' knowledge possible difficulties regarding the video explanations, and a connection between the out-of-class and in-class necessities.The work in class was based on problem-solving techniques in groups of four or five members.Subramaniam and Muniandy 19 used short videos before the class sessions.In the class, for the first hour, the lecturer prepared questions to make students recall their micro-lecture by giving interactive questions using Kahoot.Students were asked to form groups to discuss in groups with peers before answering questions given in Kahoot-It.Hussain 20 implement a flipped classroom experience with 18 students in the mechatronics subject.These engineering degree students must watch some videos with the most important concepts before the next lecture.Afterward, class sessions were composed by different techniques: engaging students by using audio and visuals aids, student-student interaction, lecturer-student interaction, problem-solving exercises and hand-on activities.Bloom's taxonomy was used for students' summative assessments after the class session.Taking into account the final academic results, Hussain et al. concluded that did not enhance significantly.Awidi and Paynter 21 supported by pre-recorded lectures, uploaded to the LMS a week before each in-class session.Class sessions were to be a time for students to interact with the lecturer and each other in order to pursue their learning goals, resolving proposed problems, and discussing matters of individual interest.The first activity was always an online quiz in LMS.After class, supplementary instructional material (e.g., videos) and other resources were provided through the LMS, including the opportunity for students to improve their knowledge through new exercises, and they had the opportunity to contact the lecturer if they required assistance.Finally, Cho et al. 22 had videos 10 to 15 min long and covered the concepts of the topics that would be discussed in the next class session.Due to the fact that watching the video does not automatically equate to students' learning, the lecturer provided them with a pre-quiz to check their knowledge.In a face-to-face classroom, the lecturer recapped the concepts from the videos, focusing on practicing problem-solving.Students were encouraged to work in groups and the lecturer facilitated the discussions throughout the problem-solving process.During in-class activities, students had more opportunities to interact with content materials and receive more timely feedback.
Nevertheless, this is not the only option.In the next contributions, there were more than one lecturer in one subject.Castedo et al. 23 checked students' level with a five-question test after each video.The length of the videos varied between 5 and 10 min.The questions were always related to the videos and mixed true or false type with multiple-choice answers.Students had to answer correctly all questions to pass, having the possibility of making an unlimited number of attempts without incurring a penalty.At the beginning of the class session, they have a 10 min link activity.The other 80 min were used for problem-solving.Students were divided randomly into groups of 4 or 5 students to solve them.A representative member of the group that finished first normally explained the completed work to his or her colleagues.Chis et al. 24 required students to study basic programming concepts in advance of the class sessions by watching a set of short videos.Students were asked to complete a quiz after each video to provide feedback to the lecturer.The classroom time was utilized for solving in-class practical exercises, and raising and answering questions.The lecturer's role was to guide students as they applied concepts and engaged with the content.In addition, they implemented the project based learning methodology to complete some class activities.Parejo et al. 25 accomplished a flipped classroom experiment in the software architecture and integration subject involving 6 lecturers and 434 students.These students are in their second year of software engineering degree.They compare the results of two different academic years.The first step was watching videos at home, and the first activity in class was answering a questionnaire about the video content.This is the way to evaluate the students' knowledge before the class session.When the students do not answer a question correctly, a new explanation is imparted to students to enhance their knowledge.Finally, Sevillano-Monje et al. 26 involved 136 students, who participated voluntarily in the study.Students must watch videos in YouTube platform before the class sessions.Students were provided with the necessary support and explanation of Cornell's method to effectively view the videos.This technique involves taking notes on a sheet that is divided into three columns or sections, namely, ideas, classroom notes, and summary, which aids in gathering the required information from the videos.At the beginning of the class, they used Kahoot to evaluate students' knowledge.The initial activity, combined with students' questions and comments during the first few minutes of the video, enables an evaluation of its effectiveness.This feedback helps determine if any adjustments are necessary or if a new video should be created to enhance understanding.Following this, students complete tasks or activities related to the content to gauge their prior knowledge and skills.
The most difficult flipped classroom contributions to find are those that involve more than one subject with more than one lecturer.Nonetheless, some of them can be found in the literature.For instance, Turra et al. 27 asked the students to watch 1 or 2 videos, with a maximum duration of 8 min.These videos are complemented with brief questionnaires or a short research activity, which allows the lecturer to monitor students' comprehension of the contents.At the beginning of the class, students discussed online activities.Afterward, the students in groups analyze the problem, discuss the implications of the data presented and reach some conclusions on how to solve it.They present the solution(s) they have encountered and the rationale they used to find them.Kim et al. 28 performed a flipped classroom experience with 115 students divided in three different groups.Ahead of the classes, they had some YouTube videos to watch.At the beginning of the class, students answered short quizzes with 3-5 multiple-choice questions.Role-plays, group presentations and video based problem solutions were also carried out.Additionally, the authors elaborated nine design principles for flipped classroom experiences with face-to-face sessions.Table 1 summarizes the information related with all previous contributions.The courses refer to the year in which the flipped classroom instructional model has been implemented."Course 0" refers to a pre-university subject.It can also be noted that Turra et al. 27 implemented it in subjects of different years.University column acronyms refer to M as Miami University, USC as University of Southern California, UPM as Universidad Politécnica de Madrid, UOW as University of Wollongong, UWA as Western Australian University, NCI as National College, LU as Lingnan University of Ireland, UCT as Universidad Católica de Temuco, SU as Seattle University, US as University of Seville, CUT  28 and Turra et al. 27 who implemented the model in three different subjects.In most cases, the approach has been implemented in a single class or in subjects that, due to the large number of students, were divided in two classes.However, six contributions have divided the experiment into two groups: one without the flipped classroom approach and another one that followed this instructional model.In these cases, the numbers in the "#Students" column are separated by a "+" symbol, where the first number refers to the class without the flipped classroom experience and the second to the group of students who followed the flipped classroom approach.Having one lecturer is the most common case, but Parejo et al., 25 for example, used six different lecturers, which is by far the contribution with most lecturers.Kim et al., 28 Turra et al., 27 and Castedo et al. 23 had three lecturers.Duration column refers to the duration of the videos, from the shorter video to the longest one in each contribution.There is no doubt that most videos are short, which facilitates the student to study about the topic.YouTube is the most common video platform to share their videos, but it is not the only one.Most questionnaire platforms are Kahoot, Moodle SV-IVR (Spherical video-based immersive virtual reality) and LMS (learning management system).The "?" character means that the contribution does not provide such information.If the cell is empty, means that they do not use quizzes in their case.

METHODOLOGY
This section aims to describe the employed flipped classroom instructional model in detail, starting from its inception and covering its different phases, as well as the methodology.

Inception and the collective nature of the experience
This experience has taken place in a Faculty of Engineering, with around 1.700 enrolled students, with previous extensive teaching experience in the use of active methodologies such as the Problem Oriented and Project Based Learning (POPBL) 29 for over two decades in all of its engineering degrees.However, from the lecturing faculty's point of view, it was becoming clearer that traditional lectures (i.e., teaching activities that take place out of the POPBL) based on oral exposition were failing to engage students, and that something was needed to be done in order to increase student participation, motivation, and improve learning experience.
This necessity was exacerbated by the global COVID-19 pandemic outbreak, in which face-to-face lectures abruptly evolved into online lectures.This further complicated student participation as a consequence of the loss of in-class spontaneous interaction between lecturers and students.
As such, with the aim of increasing the in-lecture time value, the flipped classroom instructional model was chosen as the most viable lecturing approach for the problem in hand.This choice was backed by the governing body of the Faculty of Engineering and was also included in its 2021-2024 strategic plan, devised in May 2020.The project for the implementation of the flipped classroom model presented and approved by the governing body of the faculty of engineering had as its primary objectives (i) to increase student involvement and motivation and (ii) to train lecturers in the digitization of course content.
The experience would be designed as collective in nature, it would not take place in single, scattered subjects across different degrees, but it would rather involve a significant subset of subjects within a single degree (computer science).After revision by the academic management of the faculty, the project and related resource application for its implementation in the academic year 2020-2021 were approved.In the 2021-2022 academic year, the experience continued its course, incorporating new subjects and lecturers.In its final form, the flipped classroom experience encompassed two or three academic subjects per semester and course, out of five or six total subjects in the semester.In total, 15 subjects participated in this collective experience (11 the first year and four more the second year), involving 29 different lecturers and 174 students in total (taking into account the number of students in both academic years).
After its approval, in the first half of July 2020, three main actions took place, organized and funded by the governing body: 1.The design of the experience itself (described in Section 3.2) based on the experience gathered by evaluating similar approaches, already analyzed in Section 2. 2. The design and provision of a professional-grade recording studio for the creation of teaching materials.This studio has specific high-end hardware such as cameras (intended for lecturer and document recording), a large TV, microphones, a green screen for chroma-keying and diverse lighting apparel.3. Specialized training for the lecturing participants of the experience.This training was focused on the flipped classroom instructional model and was comprised by two types of sessions: two lectures by external experts in the field of flipped classroom, who already had used it in similar settings and, in addition, a set of three workshops (the attendance of the faculty was encouraged): (a) An initial workshop describing the flipped classroom model and the set of defined general guidelines to be followed by the participant subjects in the experience.
(b) A second workshop aimed to guide lecturers on the application of the instructional model and the guidelines on their own subjects.(c) A final technical training on the usage of the aforementioned recording studio.

Description of the experience
The main characteristic of this collective flipped classroom approach is based on transferring part of the teaching experience from in-classroom lectures to a set of online videos that students can access anytime, particularly applied to theoretical concepts.In-classroom time is used for other activities, aimed at reinforcing and applying the newly acquired knowledge.
The activities comprised in the used flipped classroom instructional model can be separated in two main parts: pre-lecture and in-classroom ones.

3.2.1
Pre-lecture activities First, lecturers prepare a set of videos with the lecturing material they want to cover.In order to ease understanding and keep the viewers engaged, the created videos should be kept to a single topic and relatively short (no more than five or ten minutes per video).Complex topics should be split along several videos, if possible.
Once the material is created, the lecturer publishes the videos in a private YouTube channel.Then, the links are distributed to the students.Students are required to watch the videos before the lecture takes place.

In-classroom activities
At the beginning of the class, students complete a short quiz related to the videos they have previously watched.The quiz should be designed in a manner that allows the evaluation of the correct understanding of the concepts covered in the videos.The quiz can be built with any tool that allows for automatic correction (Moodle, Kahoot, Google forms etc.).The purpose of the quiz is twofold: students are expected to come to class with the pre-class activities completed and it also allows the lecturer to identify potential misunderstandings of the covered material in real time, by examining the results of the questions.
After the quizzes are completed, the lecturer provides feedback on the obtained results and also opens a Q&A round aiming to clarify any doubts students might have on the video material or the quizzes.
The time devoted to the quizzes and this conceptual review of the pre-lecture material should not exceed the length of a quarter of the lecture time, as the objective is to devote classroom time to activities that foster engagement and the reinforcement of the previously covered concepts.Such activities are centered in doing exercises, in-class labs and other practical activities.The aim is to maximize classroom time to reinforce the learning process by putting into practice what students have already covered in the pre-lecture phase.Some lectures have used other in-classroom complementary active teaching methodologies, such as team based learning. 30ormally, this cycle continues for every lecture.However, there are exceptions, as some concepts explained in a set of videos can require a larger number of practical in-classroom lectures to grasp them appropriately.
Students remain having access to the videos for the remaining of the academic year, so they can replay them as needed, for example, for exam preparation.

Grading
Throughout the experience, the focus has been on the transformation of the lecturing part, while student grading methodology has largely remained unchanged.The assessment of most subjects has not been modified by the implementation of the experience.This assessment is based on continuous assessment methods which employs different types of tests: such as questionnaires, practical laboratories, or traditional exams.
A small exception lies in that in addition to the aforementioned unchanged evaluation methods, in some subjects the in-classroom quizzes had a small weight on the final grade computation of the subject (1∼0%).This was done to further encourage students to review and properly work on the pre-lecture videos.

Encountered issues
The main issue from the lecturers point of view was to integrate complex ideas in bite-sized videos of optimal length (5-10 min).While in a first iteration of the experience longer videos were created (up to 30 min), this approach was fixed in the next course by using the following techniques: Multistep related but independent videos (a set of videos independently covering different aspects of the complex setting, along with a final re-cap video).This required re-engineering the scripts and splitting the content in multistep videos.
Provide the introduction in the video, while the rest of the concept was deepened using other types of media (in-class labs and activities, reading material … ) Initial longer videos also presented another secondary in-class issue: longer videos meant more content to cover and therefore, more questions were needed in order to verify the understanding of the material in one sitting, leaving little time for other activities in class, reducing the quality perception of the in-class time.However, splitting the original videos into smaller ones and organizing the watching workload in different days (with different smaller questionnaires in place) allowed to keep optimal allotments in place and having enough time for other activities.
Moreover, in the beginning, several of the lecturers planned their video watching schedule with short deadlines, to encourage video viewing the day before the lecture took place (to encourage memory freshness), this lead to spikes in video-viewing workload when several different lectures mandated viewing at the same time, rendering impossible, at times, to watch all the videos in such short notice.
Therefore, lecturers made videos available well beforehand, and also coordinated with other lecturers in order to keep such spikes to a minimum and encouraging students to organize their own watching workload.
From the point of view of the organization, the main concern was related to the initial skepticism of the student body, fearing that the off-lecture activities would take an important toll without seeing an improvement for in-class activities.This concern was overcome by substituting generic in-class exercises by added value activities, where team-based and interactive tasks the conformed the majority of the in-class time.

RESULTS
This section studies the reception of the experience by students and lecturers.The surveys conducted to gather the opinion of the participants are presented, followed by an analysis of each of the research questions.

Students' experience
We surveyed all 174 students who have participated in this new initiative to gather their thoughts on the implementation of flipped classroom instructional model.In each academic year, each student has completed the surveys for a specific course.Thus, if a student is in the second year but repeats some of the subjects of the first year, he/she is asked to respond in the surveys of the second year.At the end of each semester, they were asked to fill in a survey for each of the subjects using Google Forms.170 of the 174 participating students responded to any of these per-subject surveys (113 students in the 2020-2021 academic year and 130 students in the 2021-2022 academic year, with 73 of them participating both years).The questions, shown in Table 2, were formulated following the work conducted by Jeong et al., 31 and the responses were collected on a Likert scale. 32A summary of these responses is shown in Figure 1.

TA B L E 2
Results of the student survey (A, agree; D, disagree; N, neither agree nor disagree; SA, strongly agree; SD, strongly disagree).In addition, following the work of Jeong et al., 31 students were asked to respond to a questionnaire about the experience as a whole at the end of each academic year.All 174 students who have participated in the experience responded to this survey (120 students in the 2020-2021 academic year and 133 in the 2021-2022 academic year, with 79 of them participating both years).The questions asked in this survey (Table 3) focused on the course, rather than the specifics of each subject.In the second year, some additional questions were included to compare experiences with the previous year.These questions were only answered by students who had already used flipped classroom in the previous year (i.e., second and third grade students).Figure 2 summarizes the numerical results of this survey.

Questionnaire
The following sections answer the research questions related to students, based on the information collected in these surveys.
RQ1: Does the flipped classroom instructional model improve the students' perception of the learning experience?
Based on the results of 2020-2021 Q9 and Q10, students were able to complete the class exercises and activities with more confidence as they were familiar with the exercises and activities and did not feel lost.177 responses (53.63%) agree or strongly agree with Q9 (60 disagree or strongly disagree, 18.18%) while 192 (58.18%) agree or strongly agree with Q10 (47 disagree or strongly disagree, 14.24%).Regarding the full-year survey, 79 students responded to Q12 (65.8%) that the course was considered a "very valuable learning experience," as opposed to 6 responses (5%) that disagreed with this statement.
The results for the 2021-2022 academic year were very similar, although slightly lower than the previous year.300 responses (54.15%) agree or strongly agree with statement Q9, and only 88 disagree or strongly disagree (15.88%).In Q10, 296 responses agree or strongly agree (53.43%) and 80 disagree or strongly disagree (14.44%).In statement Q12, 78 responses (58.65%) highlighted a "very valuable learning experience" and only 16 (12%) disagreed with the statement.
As can be seen, the implementation of the flipped classroom instructional model has received positive feedback with this regard, and it can be stated that it has improved the students' perception of the learning experience.

TA B L E 3
Results of the full-year student survey (A, agree; D, disagree; N, neither agree nor disagree; SA, strongly agree; SD, strongly disagree).In the first academic year, in 233 responses to Q1 (70.6%), students agreed or strongly agreed with the statement about watching the videos before lectures, showing their commitment to the experience (only 37 responses disagreed or strongly disagreed, 11.2%).On the other hand, according to the results of Q8, the implementation of the evaluation questionnaires did not receive the same consensus, as 75 responses (22.72%) disagree or strongly disagree with this statement, while 150 responses (45.45%) agree or strongly agree.In the full-year student survey, 60 responses (50.0%) indicated that students felt that this course was more interactive than others (Q13), while only in 15 responses (12.5%) they disagreed or strongly disagreed.

Questionnaire
In the second academic year, the results are very similar to the previous ones.For the Q1 statement, 374 responses (67.5%) agree or strongly agree and, on the contrary, 81 disagree (14.62%).The results are slightly lower than the previous year, but not by a significant margin.However, the results for statement Q8 are better in the second academic year.290 answers agree or strongly agree (52.34%) and 89 disagree (16.06%).The table shows how the median has improved.For Q13, 54 responses agree or strongly agree (40.6%) and, on the contrary, 22 disagree (16.54%).
According to these responses, students' dedication and engagement has been improved in the current course due to the implementation of the flipped classroom approach.RQ3: Does the flipped classroom instructional model improve students' perception of the understanding of the subject matter?
The methodology used throughout the course has helped students to easily understand the content, as 70 of the responses (58%) agree or strongly agree with Q11 in the 2020/2021 academic year and 75 in the 2021/2022 academic year (56.4%) (8 responses disagree or strongly disagree, 6.7% the first year and 18 the second one 13.5%).Similarly, in 195 responses of Q4 (59.1%) say that the videos have helped them to learn, while only in 53 responses (16.06%) they disagree or strongly disagree in the first academic year.In the second year, 309 responses agree or strongly agree (55.77%) and 100 disagree or strongly disagree (18.05%) with this statement.Also, it is worth noting that the responses to statement Q6 were very favorable.242 responses in the first year (73.3%) and 357 in the second year (64.44%) stated that the availability of video resources was a positive aspect because they had the opportunity to revise them.
Altogether, we can state that these students' perception of their understanding of the subject has improved.RQ4: Has the flipped classroom experience been better than last year?
To answer this question, five additional questions were proposed to the students in the second year (Q16-Q20).Both, the flipped classroom experience itself (Q16) and the materials (videos, Q17 or quizzes, Q20) seem to have maintained the level of the previous year.It should be noted that students think that three subjects per semester is adequate (Q18), so we would not highlight differences with the previous year.Finally, the students think that having used the flipped classroom methodology in the previous year has helped them in the following year (Q19).
It should be noted that for all these questions, there were more "agree" or "strongly agree" responses than "disagree" or "strongly disagree." It can be affirmed that for the students the level has been maintained during the course, but that having participated in the experience the previous year has been valuable for this second year.

Lecturers' experience
The completion of a questionnaire has been carried out to evaluate the opinion of these activities by university lecturers.This questionnaire has followed the methodology used for students.In this case, the questions were defined following the work of Lai et al. 33 In 2020/2021, 18 responses were received from 18 different lecturers, while in 2021/2022, 21 lecturers participated (one of them refused to respond).11 of these lecturers repeated the experience in both years.In the second year, there were four types of teachers: 4 lecturers who were included in a subject with previous FL content, 10 lecturers who followed with their subjects, 6 lecturers who included the methodology for the first time in their subjects with no previous experience (one of them refused to respond) and one last lecturer who followed with his subject while he was included in a new one with previous FL content.At the end of the second year, these lecturers answered several more questions to find out if the effort required this second year had been more elevated, less or the same as the previous year.The statements and responses have been summarized in Table 4 and Figure 3.
Once the information from these surveys has been presented, we proceed to answer the research questions related to the lecturers.RQ5: Is it feasible to implement a flipped classroom experience at a collective level?
The faculty has identified the equipment supplied by the university as satisfactory to accomplish the experience.All of them agree or strongly agree with Q13 and Q14, and only one response (5.6%) disagrees with Q15 in the 2020/2021 academic year.The second year, only one response (5.6%) disagrees with Q13, Q14, and Q15 in the 2021/2022 academic year (5%).Moreover, 11 lecturers (61%) agree or strongly agree that the mentoring and coaching resources provided were sufficient in the 2020/2021 course and 14 the second one (70%).On the other hand, 17 lecturers (94.4%) have agreed or strongly agreed that completing the flipped classroom instructional model's tasks required additional time to complete (Q11) the first year and 15 the second one (75%).This decrease could be due to the fact that in the second year, a part of the material was already done in the previous year.

TA B L E 4
Results of the lecturer survey (A, agree; D, disagree; N, neither agree nor disagree; SA, strongly agree; SD, strongly disagree).After implementing this flipped classroom experience at a collective level, lecturers are favorable to continue using this instructional model, according to the favorable responses in Q17, Q18, and Q19 with 13 (72.2%),12 (66.7%),and 13 (72.2%)lecturers agreeing or strongly agreeing respectively in the 2020/2021.The second year results have improved with favorable answers to the same statements Q17, Q18, and Q19 with 18 (90%), 14 (70%), 15 (75%) respectively.These results show the feasibility of implementing the flipped classroom experience at the collective level, as long as enough resources are provided.

Questionnaire
RQ6: Does the flipped classroom approach model improve the teaching experience?
According to the results, the teaching staff enjoyed the flipped classroom experience and appreciated the novelty of this instructional model, as none of them disagreed with statements Q1 and Q2, while 13 (72.2%)and 12 (66.7%)lecturers agreed respectively the first year and with 14 (70%) and 12 (60%) the second one.In Q6, 13 lecturers (72.2%) agree or strongly agree that they are unaware about the recognition that they can obtain and, the second year, it decreased to 9 (45%).In Q8, only 5 (27.8%) were concerned about the recognition mechanisms that exist from the flipped classroom approach community, and it also decreased the second year to 3 (15%).These results show that lecturers are more motivated by the challenge than by the potential compensation.
All in all, it can be said that the flipped classroom approach has contributed to improving the teaching experience.RQ7: Is the effort to prepare the flipped classroom sessions greater the second year?Questions Q23, Q24, and Q25 were for lecturers who used someone else's material.The questions want to find out if the effort of having used someone else's material and flipped classroom has been greater.Only one lecturer believes that preparing the class sessions (Q24) and quizzes (Q25) have been more effort.For all others, teaching (Q23, 100%), preparing class sessions (Q24, 80%) and quizzes (Q25, 80%) have involved less effort.
As for questions Q26-Q30, they have only been answered by lecturers who have repeated the subject.The effort to prepare classes (Q26, 9 lecturers, 81.81%) and teach (Q27, 8 lecturers, 72.72%) was less.In addition, 54.54% (6 lecturers), believe that it is easier to prepare the sessions used in the previous year (Q28), and also to prepare new sessions (Q29, 7 lecturers, 63.63%).On the other hand, preparing new videos was not considered easier (Q30).4 lecturers were in favor (36.36%) and 3 were against (27.27%).
It can be stated that the effort required to prepare previous sessions, teach and prepare new material is more effortless the second year than the previous year.

DISCUSSION
This article presents how the flipped classroom approach has been successfully incorporated into the computer science degree.Although not all subjects have adopted this methodology, those that have, driven by motivated faculty, have achieved favorable results, as reported by both students and faculty.Admittedly, the core faculty had to invest additional effort to facilitate this transition, but the increase in student satisfaction makes it a worthwhile endeavor.
In the literature, most contributions refer to experiences of one subject with one lecturer.It is more difficult to find single-subject experiences involving more than one lecturer.Undoubtedly they exist, but they are fewer.It is much more difficult to find experiences involving more than one subject and more than one lecturer.In the experiences found in the literature, the subjects belong to the same course.No collective experiences have been found that implement the flipped classroom experience in more than one course at a time.For this reason, we believe that this experience contributes a new aspect to what already exists.
The implementation of the flipped classroom methodology carries significant implications, including heightened student engagement, optimized classroom time, personalized learning experiences, increased student accountability, and improved assessment methods.For faculties, the key implications involve re-evaluating their lesson plans, creating high-quality learning materials, offering student support, fostering collaborative learning, and prioritizing assessment strategies.In general, flipped classroom necessitates a more student-centered approach to teaching and learning, and instructors must be adaptable and attentive to the needs and interests of their students.
Although it can be a beneficial and effective approach, successful implementation requires thorough planning and consideration of potential risks, such as student disengagement, inadequate faculty preparation, and limited interaction.Therefore, it is crucial for faculties and teaching institutions to be aware of these risks and take steps to address them, such as providing guidance and support to students who struggle with self-motivation.
Ultimately, the efficacy of flipped classroom will depend on various factors, including student academic level, subject matter, and instructor goals, all of which should be carefully assessed when deciding whether and how to implement the methodology in the classroom.

CONCLUSIONS AND FUTURE WORK
In this contribution, we report the findings of applying the flipped classroom pedagogical approach in the computer science degree.The experience is motivated by two factors: (i) the change in the information consumption habits of the new generations entering higher education and (ii) a methodological evolution owing to the difficulties performed during the pandemic of the 2019-2020, which were determinant in the needed stimulus to enhance the teaching experience.The experience carried out has involved all the students of courses 1, 2, and 3 of the computer science degree during the 2020-2021 and 2021-2022 academic years.In total, 29 lecturers and 174 students (some of the students participated in both years) in 15 subjects were involved in the experience.The design of the experience, its coordination and implementation as a team (i.e., team of lecturers), has led us to call it a collective experience.
The experience included a first phase of procurement of materials and infrastructure, as well as training for the lecturers involved.In a second phase (i.e., during the teaching of the subjects), more than 250 videos and more than 80 questionnaires were generated.
Three referenced survey models were used to evaluate the experience.Two of them were addressed to students and the other one to lecturers.With the surveys, the students have evaluated on the one hand the flipped classroom experience of each subject and on the other hand they have evaluated the experience as a whole.Based on the results obtained, it was found that the students' perception in relation to (i) the learning process, as well as (ii) dedication and motivation, and (iii) the level of understanding of the subject have improved thanks to the flipped classroom experience.These results allow us to conclude that student motivation has improved in the subjects that have adopted the flipped classroom pedagogical approach.
In relation to the results obtained from the faculty survey, it is concluded that the faculty considers that the teaching experience has improved and is in favor of continuing with the experience in future courses.We consider that this path taken is correctly aligned with the objective of training lecturers in digitization of subjects.
The faculty team involved in the experience has also noted the importance of several factors to ensure the success of the project.On the one hand, it is essential to provide instruction in the pedagogical approach as well as training in the different tools (i.e., video recording/editing, devices, etc.).It is also important that the lecturers involved have a constructive attitude for change.Finally, it is fundamental that the institution supports the project with resources so that the work can be carried out with the proper quality.It should be noted that the effort required in the second year is considered to be reduced.
In the future, we are interested in improving and enhancing the experience.On the one hand, the feedback collected in the evaluation surveys of each of the subjects has allowed us to identify a number of actions for improvement in relation

Frequencies 1
video material was well-designed, well-structured and clearly defined.activities developed in class after the quiz have helped me to learn.ability to rewatch and rewind the videos has helped me to learn.short video format presenting the main study topics helped me learn more than the very detailed and extensive videosmultiple-choice quizzes after watching the videos have allowed me to delve deeper into the more complex content before class and therefore helped me understand it better.watched the videos and reviewed the materials provided prior to the class sessions has helped me to complete the class activities with more confidence as I was not at a loss.watched the videos and reviewed the materials provided prior to the class sessions have helped me to complete the class activities more easily because the activities were familiar to me.Results of the student's survey (A, agree; D, disagree; N, neither agree nor disagree; SA, strongly agree; SD, strongly disagree).

Frequencies 2
flipped classroom instructional model used in this course has helped me to easily understand the content.would like to continue with the same flipped classroom instructional model the next few years.model used in this course is useful to be applied in other subjects.you think the use of flipped classroom has been better than last year?you find the quizzes after the videos easier?Results of the full-year student's survey (A, agree; D, disagree; N, neither agree nor disagree; SA, strongly agree; SD, strongly disagree).RQ2: Does the flipped classroom instructional model improve students' dedication/engagement?

Frequencies
is the driving force behind much of what I do in flipped classroom experience.long as I can do flipped classroom experience, I'm not that concerned about exactly what recognition I can obtain.efficacy Q9: I could complete the flipped classroom instruction model task if there was no one around to tell me what to do as I go.could complete the flipped classroom instruction model task if I could call someone for help if I got stuck.could complete the flipped classroom instructional model task if I had a lot of time to execute flipped classroom approach.have sufficient ability to prepare teaching materials for the flipped classroom instructional model tasks in advance (such as recording videos and collecting educational resources on the Internet).I could, I would like to continue my use of flipped classroom instructional model.think my students would be in favor of utilizing flipped classroom approach in their class.think my students would believe that flipped classroom approach could be a useful educational method in their class.it taken you more effort than usual to teach the classes that the previous lecturer prepared last year with flipped classroom?it taken you more effort than usual to prepare new sessions with flipped classroom than to use the lecturer'it taken you more effort than usual to use the previous lecturer flipped classroom surveys to prepare the sessions?it taken you more effort to prepare the classes than you did last year with flipped classroom?it taken you more effort to teach the classes that you prepared with flipped classroom last year?it easier for you to prepare last year's classes with flipped classroom?
Analysis of different flipped classroom instructional model implementations.
TA B L E 1as Chalmers University of Technology, USM as University Sains Malaysia, MWU as Midwest university and CUHK as Chinese University of Hong Kong.Most contributions have implemented in a single subject, with the exception of Kim et al.