On-line delivery of marine engineering courses during Covid-19 pandemic

1. Introduction

If there was one single event in recent history that drew the entire world’s attention after World War II, it must be the Covid-19 pandemic in March 2020. When the World Health Organization (WHO) declared it as a “pandemic” on March 12, 2020, world leaders and the media focused their attention on the unprecedented coronavirus lockdown. In London, UK Prime Minister Boris Johnson declared “it is a national emergency” as he placed the United Kingdom on lockdown, quotes the newspaper “Guardian” on March 24, 2020 [1]. Apart from the suffering and deaths caused by Covid-19, the global reaction experienced by people due to disruptions in their daily lives as a result of these lockdowns need to be investigated and analyzed.

This research aims to study the consequences of online delivery of Marine Engineering training, which was imposed on Maritime Education and Training (MET) institutes prematurely due to Covid-19 restrictions. In general, if delivered under proper guidance in certain disciplines, online education and assessment offers its own advantages and merits. However, if we adopt “one size fits all” policy toward online education without considering learner abilities and teaching techniques, we will end up with substandard results.

In marine engineering curriculum prescribed in the International Maritime Organization (IMO) STCW Regulations in Chapter III and Tables A-III/1, A-III/2, and A-III/6 there are subject units that can be taught and assessed:

• 1. Completely online
• 2. Completely face-to-face (Lab and simulator practicals)
• 3. A combination of online and face-to-face

During the pandemic, only the category 2 and 3 engineering subject units constituted an issue that required immediate solutions for MET Institutions and their teaching staff. This issue was addressed by most MET institutions by implementing modern technology. Video conferencing software applications like Zoom or Microsoft Teams. were used to replace face-to-face instruction in the classroom. These apps quickly acquired popularity among students and lecturers due to their ease of use and plethora of facilities.

2. Hard Facts about Teaching Online

At the beginning of the pandemic, MET lecturers were quite happy with the capability of these software applications in a virtual classroom. The lecturers were able to substitute the classroom white board with electronic writing surfaces and demonstrate in front of the webcam. The stage was set, the lights and cameras were turned on, recording ready, and the Power Point presentations began. We, as lecturers, expected to confront the challenge of modern technology and for the delivery to go well and the knowledge transfer to be successful. However, there is one critical aspect in the whole exercise that has been overlooked. It is the audience or the lack of it in a virtual classroom setting. We quickly realized that the students were not in front of us, but rather in the comfort of their own homes; at best, in their bedrooms, or at worst, in the control room of a ship keeping a watch amid reasonable disturbances.

Normally, marine engineering operational units (viz. Engineering Knowledge, Marine Electrical Engineering, and Control Systems Engineering) must be taught face-to-face so that the lecturer can check whether the communication channel between himself and the students is intact by questioning the students and receiving immediate feedback.

“Feedback” is an essential component of effective learning and enhancing students’ learning experiences. Student feedback has become a popular way to evaluate and improve teaching effectiveness. Feedback from your students allows you to see what you are doing that facilitates learning, as well as what changes need to be made. Even minor changes in the classroom can make a huge difference [2]. The working principles of various marine machinery and details of design and operational variables and malfunctions, are covered in these units, which require creative and cognitive thinking. Another way to receive feedback is to observe students’ body language when they are physically present in front of you. This does not happen with online delivery since most students keep their cameras and microphones muted. Some students prefer to keep their cameras on, but the majority of them switch them off from the start for several reasons, including data storage and privacy issues. Others turn off cameras owing to technical issues, such as weak internet connections, so that they can hear the session uninterrupted. The ultimate result of not being able to see the students in front of us is the complete breakdown of the interaction between the lecturer and the students. There is simply no “eye contact” with your students. It is further exacerbated by computer glitches and time delays. When communication is absent, it is difficult to accept that online education is the best for marine engineer training. In a virtual classroom, the regular real-world classroom environment is absent.

Although we have realized the complete breakdown in communication since students do not respond to our questions and comments, we are helpless because the time allocation for online teaching cannot be increased to re-establish the communication channel by giving more time for students to interact. In an online form of delivery, all students who gave feedback agreed that there was a severe breakdown in communication between the lecturer and the students, as well as among the students. As a result, the teaching staff decided to have a 15-minute question and answer session every hour to keep students engaged and encouraged them to interact with the course based on their feedback. This will be implemented in the next delivery if the issue persists.

❖ Students’ comments on lack of interaction between the lecturer and the virtual class:

• a) Effective face-to-face casual/informal discussion is absent from online learning.
• b) There is no group discussion in online learning, whereas on campus students can do so.
• c) As students will be taking courses online, they will require WIFI/broadband/mobile network, but some countries’ internet connections are poor. As a result, students faced numerous problems during their studies and exams.
• d) Face-to-face interaction is severely lacking.

3. Hard Facts about Learning Online

The negative effects of online learning have an impact on students’ lives and expectations in many ways. In the following paragraphs, we will present some of the ways that this situation contributed to their frustrations and disruptions of their well-being.

In a large country like Australia, most students need careful planning before moving interstate to study at UTAS/AMC. They must plan their family matters after negotiating with their shipping companies for study leave and sponsorships. When students from mainland Australia move to Tasmania and settle down with their families for the study and assessment period of the course at UTAS/AMC, they may need to sell or rent out their houses, as well as sell their family cars. They make life-altering decisions at these junctures in their lives, which are critical and must deliver the right results. During the pandemic, those of them who had planned their studies and taken leave from the shipping companies were wondering why they need to be confined to their rooms at home and learn online when they had expected to be on campus and learn face-to-face.

During the orientation program, some of our very senior and experienced engineers, whom we respectfully consider as “mature students,” stated that they prefer “conventional classroom learning and hardcopy study guides and printed textbooks,” over electronic study material, because their brains are hardwired for on campus face-to-face learning rather than online learning. The students mentioned that the continuous use of the computer screens for prolonged periods of time is physically more exhausting than attending an on-campus lecture. These students stated unequivocally that they prefer written books over looking at a computer screen. This might have adverse effects on physical and mental health of the students.

They expressed their dissatisfaction and anger at the suggested internet delivery orientation. During the first two weeks of the semester, cracks appeared on their learning foundations as they grappled with new electronic blackboard* and online learning. This resulted in a flood of email enquiries from students seeking clarifications on various study-related issues. In a classroom setting, these issues would have been resolved in a matter of minutes. Some of the frustrated students withdrew from the course, while others simply “gave-up” their studies until face-to-face classes returned.

4. Inability to do Practical Components of the Course

Another major drawback of online engineering courses is the inability to complete the practical components, which requires students to be present on campus. Although video demonstrations and additional materials are used, it is quite difficult to bridge this gap.

MET Institutions normally draw a large number of international students, and most countries’ travel restrictions have rendered the international travel almost impossible. While the travel restrictions are in place and it is evident that international students cannot be on campus to attend classes, MET Administrations insist that practicals be conducted as they were before the Covid era. Due to these strict regulations, MET Institutions are restricted from enrolling overseas students for future intakes. The online assessment is by far the most critical shortfall in online delivery. This will be discussed in detail under sub-heading # 6 of this paper.

5. The Satisfied Customers

Although the majority of students did not approve of online teaching and learning for marine engineering cohorts, there were a few others who praised it.

Through the feedback survey, the students commented on the positives of online delivery as indicated below:

• a) Most students agreed that online learning is financially beneficial for them since they can stay at home and avoid paying for accommodation, travelling to campus, and free WIFI is available at home. Since the cost of accommodation is quite high in both Australia and Japan, it is a considerable saving for them.
• b) In online learning, all lectures are recorded live and then made available to students via an electronic blackboard(MyLO*/BEEF**). These recordings are important, and students can study them anytime and anywhere as convenient. Students have commented that this is highly beneficial for their learning. *MyLO—My Learning Online at AMC/UTAS,**BEEF—Basic Education Environment Frontier at Kobe University
• c) The most prominent advantage of online learning is the ability to study in a comfortable environment (mainly at home), however, this can also be a drawback if this usual environment (at home) prompts the student to adopt a relaxed or lethargic attitude.
• d) Online delivery allows students (and teachers) to save a significant amount of time, particularly in places where the commute to and from a campus is long. Once again, how this time is utilized is entirely dependent on the individual.
• e) The most pro-online comments came from students who are taking the course while working. Naturally, they were appreciative of the online delivery because it allowed them to earn while also learning, without any fear of losing the job during the pandemic. They also stated that their employers were pleased as they were not obliged to sponsor them.

❖ Comments from students praising online MET delivery:

• a) I believe that online learning is the way forward in this covid era and beyond. It allows people the flexibility to work and study simultaneously.
• b) The ability to access lecture recordings at any time and download them is a huge relief for students struggling with time zones.
• c) Surprisingly, I felt more comfortable during the exam using the lockdown browser. It helps you save time.
• d) Nothing further to add, just keep it online. It was long overdue, and it is disappointing that it could not have been done sooner. I understand that we are driven by the enormous bureaucratic wheels of Academia and Regulators, but I am really pleased with the progress that has been made. The positives of doing it online far outweigh the negatives.
• e) As overseas students, we are not required to attend classes on campus; as a result, we stay in our home country, saving money and giving students more flexibility.
• f) While studying online, students can spend more time with their families. The enjoyment is beyond expressible, for those who work away from their families for their livelihood.

6. On-Campus vs. Online Assessment: Critical Factors

IMO STCW recommends various assessment procedures for determining the competence of marine engineers seeking qualifications. The STCW Code Tables A-III/1, A-III/2, and A-III/6 in their Column 3 indicates methods for demonstrating competence in the following ways.

Examination and assessment of evidence obtained from one or more of the following sources:

• 1. approved in-service experience
• 2. approved training ship experience
• 3. approved simulator training, where applicable
• 4. approved laboratory equipment training [3]

While almost all MET institutes follow the STCW model courses for training and assessment, the contents of each subject module (unit) may vary for a variety of reasons. Regardless of the subject matter variations, the foundation and structure of engineering disciplines require a certain pattern of examinations and assessments.

For engineering units, most MET Institutes use summative assessments rather than formative assessments. At AMC and Kobe University, both the assessment regimes are employed for Engineer qualifications. Unlike other non-engineering disciplines, engineering knowledge assessment entails thorough examination of creative and cognitive thinking of the student. This, in turn, requires the student to display his rational thinking and application of engineering principles to a given problem backed by mathematical reasoning and demonstration of engineering skills and design. For example, a watch-keeping engineer can be assessed on his ability to parallel an incoming generator with a running generator. The first part of the question requires him to write the steps that he must follow. This part usually has a standard response that he learns in class and frequently practices on-board. The second part assesses his knowledge by questioning what happens if he does not follow the exact procedure. If this has been taught, he will know the answer and may write it. If it was not taught, he may assume that the paralleling procedure will fail, and the objective will not be reached. The third part of the same question requires him to demonstrate his understanding by explaining the theoretical concepts behind why, if there is a flaw in the procedure, the incoming generator cannot run parallel with the running generator. This type of evaluation cannot be formative as explanations are necessary with sketches, engineering formulae, and other supporting materials. For this reason, we always use essay-type summative questions in engineering examinations.

Prior to the Covid pandemic, MET institutions conducted face-to-face invigilated summative written examinations for all Engineering units except for the engineering skills units. However, it was difficult to hold online examinations with same level of rigor and authenticity. The MET Institutions deployed various online platforms to conduct examinations with restricted browsing of students’ computers and remote invigilation to avoid plagiarism by students.

While platforms such as Lockdown Browser, Intelligent Remote Invigilation Service (IRIS), and Respondus Monitor are equipped with the required restrictions and monitor students' movements, they cannot necessarily ensure that the restrictions and regulations of an examination will be followed.

Despite the intensive design and programming of these monitoring software, there are still blind spots where movements are not detected as much. Students have a proclivity for identifying and capitalizing these blind spots.

Furthermore, when submitting supplementary material for examination responses, (which cannot be typed in the space allotted due to engineering formulae and hand-sketched diagrams), students must scan and submit the e-copy. After the examination session has concluded, a time window is allocated for this. A larger window of additional time (allocated for technical considerations) usually results in it being used for dishonest activities.

Because the said procedures significantly rely on the student's sincerity, it is impossible to guarantee that a student's answer to an online examination is a genuinely individual attempt without collaboration or collusion using the currently established remote monitoring systems and examination procedures.

As a solution to plagiarism associated with online closed-book assessments, open-book assessments were trialed. Two mid-term tests of the same unit were conducted and compared. The first test was a closed-book invigilation and the second was open-book. The second test was designed to be an open-book and administered to the same group of students without any invigilation. Students were allowed to refer to their notes and textbooks without any restriction. The need to invigilate was not a factor as answers to the questions cannot be derived from notes or textbooks. When compared with the first test, which was done under invigilation, there was no significant difference between the class averages in the two tests. In fact, 60% of students scored nearly the same grade on both tests.

7. Research conducted at Kobe University

To determine the impact of learning Engine simulator both online and face-to-face, Kobe University recently conducted research. This research involved 19 students spanning five months. Author conducted 15 simulator lessons as Zoom meetings. Each Zoom session was an explanation of a task and a step-by-step demonstration of how to prepare and start the main engine. This phase of the research is known as “Before Face-to-face.” During this phase, students use the knowledge gained from online learning to complete the stimulator exercise. A questionnaire was given to the students at the end of each task. The questionnaire comprised NASA Task Load Index [4] parameters such as Physical demand, Mental demand, Temporal demand, Own performance, Effort, and Frustration.

Following this phase, 3/4 students were invited to the engine simulator room for a face-to-face demonstration and were given the task of preparing and starting the main engine. This phase is called the “Face-to-Face” period. The questionnaire was given to the students in this period. In the next session, the student group that completed the exercise was replaced, and the first group was allowed to watch them via webcam.

After all the activities were completed, the questionnaire was presented. This phase of the research is called “After Face-to-Face.”

A questionnaire survey was conducted to determine how the students felt about the hardship of completing the exercise tasks to achieve the stipulated result. Six questionnaire items, which are according to NASA's task load index (TLX) as subjective workload evaluation, were considered for the survey, i.e., physical demand (PD) and mental demand (MD), temporal demand (TD), and own performance (OP), effort (EF), and frustration (FR). In the questionnaire, the author asked “which is the stronger item of burden in two choices as paired comparison?” Because there are six evaluation items, the author asked 15 pairwise comparison questions from participants in a questionnaire. For each evaluation item, the author and students agreed on a scale of 1 to 7, with 7 being the strongest and 1 being the weakest. All 19 students completed the questionnaires, yielding a total of 127 sets of answers. For analysis of the answers to the questionnaire, they were categorized into three situations: before, during, and after the face-to-face exercise. The agree level is calculated as follows. Each item selection rate, which is the number of times an item was selected in 15 pairwise quizzes, was weighted. These weighted factors are multiplied to the rating of each item. The agree level for “burden is small” was computed using the sum of six items. The results of the analysis is presented in the table and figure below.

Table 1:

Average of weighted agree level

Figure 1:

Results of Thurston's paired comparison method

The table shows the average of the weighted agree level, which is determined based on the participants answers. The agree level seems to be increasing from the beginning of the face-to-face exercises to the end. Thurston's paired comparison method is depicted in the figure. Thurston's paired comparison method assigns relative weight to evaluation items on the first-dimension graph. Following face-to-face exercises, we observe two changes in online lessons: the mental demand tends to increase, but the physical demand tends to decrease. We presume that face-to-face knowledge of the exercise task and observing the exercises of others’ performance from a bird's-eye view via web camera changed the consideration load for the exercise task.

In this specific technique of employing simulator as a teaching tool, we should consider combining online delivery and face-to-face training for optimum results. This perspective could lead to more effective maritime education in general and use of Engineering simulators.

8. Conclusions

As previously stated, online education was pushed upon all types of educational institutions as the sole solution to complete courses that were started before the Covid pandemic. Due to the magnitude of the issue that warranted unprecedented solutions, MET institutes adopted various contingency measures such as online web conferencing software for teaching and a variety of software for assessment. The problems experienced by both students and lecturers were extremely similar across AMC and Kobe University.

• 1. While analyzing the complaints of students during this period, both institutes found that students contacted unit coordinators more frequently than during the non-Covid period. The main reason for contacting the unit coordinator was the shift from face-to-face to online delivery.
• 2. Staff at both MET institutes agree that their academic materials were not ready for online delivery at the beginning. We will need to invest some resources and expertise to develop suitable interactive and online content.
• 3. Socially interactive learners are engaged learners (Vacca et al., 2011). Routman (2005) argues “students learn better when they are able to converse to one another and participate actively” (p. 207). In short, social interaction is vital to the learning process [5].
• To keep students focused on the lesson and encourage interaction, AMC marine engineering staff recommended 15 minutes of question/answer student engagement session for every hour of online delivery. It could also be casual/informal communication between the lecturer and students, as well as among students.
• 4. In terms of assessment, the AMC staff decided on open-book online examinations rather than closed-book remotely invigilated examinations. Exams must be tailored to the competencies being assessed, leaving no possibility for students to plagiarize. We must understand what we learn in open-book tests. We used to just memorize without understanding in closed-book tests. Why? We must apply logic and common sense to what we have learnt. We must consider more than just what we have learnt [6]. Researchers compared closed- and open-book examinations in Health Science and concluded that a combination of both would be the best as it had acceptable reliability and the scores of the open- and closed-book sections were consistent in terms of students ranking. Contrary to popular belief, open-book tests were not easier than closed-book tests. Although several aspects of open-book testing need to be investigated further, this study demonstrated that open-and closed-book tests can be used together to compliment today’s assessment programs [7].
• 5. Another important facet to this issue is the inability to undertake practicals, which was mentioned under topic 4 of this document. Under the present Covid-19 situation, it is still difficult to predict when the entire world will be able to function like it did before the pandemic. As it is necessary for the students to gain this competence, the most probable solution for those who are unable to perform the practicals owing to travel restrictions is to complete a training record book on-board a ship. It will be similar to the Cadet training record book with advanced practicals. The work undertaken must be confirmed by the ship’s Chief Engineer. The creation of an advanced engineering competence record book necessitates meticulous planning, as it needs administration approval in lieu of the practicalsdone at a MET.
• 6. Taking the pandemic and its impact holistically, AMC and Kobe researchers wish to highlight the suggestions brought forward by Emeliza T Estimo and Roberto Neal S. Sobrejuanite for expanding alliances and external partnerships with global maritime education and training institutes for resource sharing, best practices, and benchmarking to face a future similar event [8].

Author Contributions

Conceptualization, T. Miwa and G. Lokuketagoda; Methodology, T. Miwa; Software, T. Miwa; Formal Analysis, G. Lokuketagoda; Investigation, T. Miwa and G. Lokuketagoda; Resources, T. Miwa and G. Lokuketagoda; Data Curation, T. Miwa; Writing-Original Draft Preparation, G. Lokuketagoda; Writing-Review & Editing, G. Lokuketagoda; Visualization, G. Lokuketagoda; Supervision, T. Miwa; Project Administration, T. Miwa and G. Lokuketagoda; Funding Acquisition, T. Miwa and G. Lokuketagoda.

References

• The London Guardian, https://www.theguardian.com/world/2020/mar/24/a-national-emergency-how-the-uk-papers-covered-the-coronavirus-lockdown, , Accessed October 15th, 2021.
• Ellison Education, https://www.ellisoneducation.com/blog/the-importance-of-student-feedback/, , Accessed October 20th 2021.
• IMO STCW Convention and STCW Code, including 2010 Manila amendments, Third Consolidated edition, pp.143, 2011
• S. G. Hart and L. E. Staveland, “Development of NASA TLX (Task Load Index) results of empirical and theoretical research, Advances in Psychology, vol. 52, pp. 139-183, 1988. [https://doi.org/10.1016/S0166-4115(08)62386-9]
• B. Hurst, R. Wallace, and S. Nixon, The impact on social interaction on student learning, Reading Horizons: A Journal of Literacy and Language Arts, vol. 52, issue 4, pp. 376-398, 2013.
• T. V. Eilertsen and O. Valdermo, Open-Book Assessment: A Contribution to improved learning?, Pergamon, Studies in Educational Evaluation, vol. 26, pp. 91-103, 2000. [https://doi.org/10.1016/S0191-491X(00)00010-9]
• M. Heijne Penninga, et al., Open-book Tests to Complement Assessment programmes: Analysis of Open and Closed-book Tests, Advances in Health Sciences Education (2008), vol. 13, issue 3, pp. 263-273, 2008. [https://doi.org/10.1007/s10459-006-9038-y]
• E. T. Estimo and R. N. S. Sobrejuanite, Remote Instruction: Challenges, Initiatives, and future directions for maritime education institutions in a developing country, Proceedings of The International Association of Maritime Universities (IAMU) Conference, pp. 64-73, 2021.