This study is a comparison group study conducted to compare the effects of video-based training on maintenance work, along with the existing theoretical education for third-class engineers of a ship. It was conducted for engineering trainees in the third grade of high school, who had no onboard experience, boarding Hanbandoho, a merchant training ship of the Korea Institute of Maritime and Fisheries Technology. They were divided into two groups, that is, 36 control and 38 experimental groups, to delve into the educational effectiveness of utilizing videos. The video used was the maintenance video footage of an engineer running and maintaining the equipment installed on the ship. This equipment is the same as that for trainees to practice group maintenance with, and was built for the ship in accordance with the NCS learning goals, as shown in Figure 1(a).

Figure 1: 
Video production and lecture of refrigerant charging

Five devices, that is, a boiler, freezer, fresh water generator, air conditioner, and oil filtration machine, were selected as the subjects to be used for education, the specifications of which are listed in Table 1. The experimental group conducted lectures using PowerPoint presentations, watched videos on each device, and operated and maintained the devices at the site. After providing theoretical lectures and references using the PowerPoint presentations that were previously implemented, the control group was directed to conduct practice. Both the experimental and control groups were given identical PowerPoint presentations, references, and practice content.

The pre-questionnaire was prepared prior to the implementation of the training, and the post-questionnaire was prepared after the device maintenance practice training was completed. The post-questionnaire was conducted in the same manner as the prequestionnaire.

The questionnaire used in this study was composed by researchers with more than five years of onboard experience as merchant ship engineers through the selection of 25 items among the learning goals of the NCS competency units in consideration of the operation and curriculum of the training ship. They were classified into five devices (E1 to E5), of which third-class engineers are in charge, such as a boiler, a freezer, a fresh water generator, an air conditioner, and an oil filtration machine, and theoretical and practice items were designed as detailed sub-surveys of five questions, respectively, as shown in Table 2.

In the auxiliary boiler (E1), the type and operating principles of the boiler burner, theoretical items of whether the burner trip and alarm items can be understood, and practical items on manual boiler operation, water test, and blowdown operation measures were constructed. In the freezer (E2), both a theoretical item that can explain the freezing principles and freezing system and the names and roles of each part, as well as practical items such as starting and stopping the freezer, refrigerant supplementation and shift operation, and lubricant supplementation and exchange. In the fresh water generator (E3), both theoretical items, ranging from the principles of the fresh water generator to the names and roles of each part, and practical items regarding starting and stopping the fresh water generator, adjusting the tidal volume and salinity, and checking and adjusting safety devices. In the air conditioner (E4), both the theoretical part of a heating ventilation and air conditioning (HVAC) and an air conditioner unload system, and the practical part covering starting and stopping the air conditioner, methods to regulate on-board temperature, and methods to exchange filters of the HVAC were designed. In the oil filtration machine (E5), theoretical parts including international law on oil filtration devices and the operating principles, practical parts concerning starting and stopping oil filtration devices, methods for testing a 15 PPM alarm device, and methods for testing an oil detector. The questionnaire was designed on a 5-point Likert scale, with one point for "not at all," two points for "not so," three points for "somewhere in the middle," four points for "generally yes," and five points for "very much so."

Reliability analysis was conducted to verify the consistency of the questions in the questionnaire. The reliability is mainly determined by calculating the Cronbach's alpha coefficient (Cronbach's α), and in general, if it is 0.7 or higher, the reliability is judged to be good. Cronbach's α for the 25 questions of the questionnaire was higher than 0.7 (0.930), and the reliability of the main variables in this study was judged to be good. Therefore, it was determined that there were no questions that undermined reliability, and the analysis was conducted without excluding any of the questions. The evaluation results from the reliability of the questionnaire by detailed competency unit are shown in Table 3. The auxiliary boiler (E1) recorded the highest points of 0.935, and the air conditioner (E4) recorded the lowest points of 0.790, which is still a significant value.

Data collected through the survey were analyzed using the SPSS 19.0 statistical program, and statistical verification of the results was conducted at a significance level of ≤ 0.05. The homogeneity test between the two groups was performed by analysing the survey data prepared prior to training. After training, the questionnaire data were analyzed, and the difference between the two groups was identified. Furthermore, a one-way analysis of variance (ANOVA) was conducted to ascertain the effectiveness of the video-applied education by equipment, and post-analysis was performed using the Tukey HSD method.

Prior to the implementation of this curriculum, a t-test was performed on the survey response results by subject to test homogeneity between the experimental and control groups during the education on ship familiarization and safety.

The results of the homogeneity test by subject between the experimental and control groups are shown in Table 4. Homogeneity verification was confirmed as there was no significant difference between the two groups in terms of the education levels of the auxiliary boilers (t=0.028, p=0.978), freezer (t=-0.115, p=0.909), fresh water generator (t=0.104, p=0.917), air conditioner (t=0.547, p=0.586), and oil filtration machine t=-0.199, p=0.843).

After the training was completed, the average difference verification by each item between the two groups was conducted to compare the practical education using the video and classroom training. To compare the detailed items of the device group, the average difference verification was performed by dividing it into five detailed items by group.

In the auxiliary boiler category, in particular, in the category of understanding both the boiler burner and its operating principles (t=1.750, p=0.085), and the boiler burner trip and alarm (t=1.916, p=0.059), the training combined with video footage did not make a significant difference compared to paper-based training. However, there was a significant difference in practical items such as manual boiler operation, water test, and blowdown (p<0.05). The auxiliary boiler of a vessel is difficult to install at a hands-on practice center in school because of the initial costs for building a facility and the difficulties in establishing relevant equipment for environmental pollutants that are pumped into the air during operation. Thus, pre-practice education was conducted mainly on theoretical education, and theoretical education was conducted as repetitive learning even onboard a vessel; thus, the video-applied training was found to be much more effective because the practice of operating a boiler was first conducted.

As for the freezer, there was a significant difference between the two theoretical and three practical items (p<0.05). As for the alternating operation items of a freezer among the practical training items, there was a significant difference (t=7.213, p=0.000), and the experimental group (M=4.868) was higher than that of the control group (M=4.222). As for the practical training item of operating a freezer, the effectiveness of video-applied education was found to be noticeably prominent because it is not possible to operate the entire system of a freezer, although theoretical aspects compared to an actual device could be understood as each system module of a freezer was installed at a practical training center of maritime schools.

As for the fresh water generator, there was also a significant difference in all items (p<0.05). Similar to other devices, there was a significant difference in the three practice items; in particular, the experimental group (M=4.316) was much higher than the control group (M=3.722) in the theoretical part of each of the fresh water generators (t=5.504, p=0.000). Although fresh water generators are installed and theoretical education is conducted at on-land practical training facilities at schools, various types of fresh water generators such as the plate and tube types of low-pressure evaporation, and reverse osmosis types used in merchant vessels are difficult to install on land, it is conjectured that a vast difference in the names and roles of each device occurred as video-applied training was only conducted onboard a vessel.

Significant values were found for the five items of an air conditioner (p<0.05). This was due to the fact that, although each device of a freezer can be observed during practical training at school, it is difficult to install HVAC systems such as an air handling unit owing to space constraints, the first hands-on operation of an actual machine onboard a practice vessel made such a huge difference.

In the oil filtration machine, there was a significant difference in all five items (p<0.05). Operating a machine after watching a video clip depicting the operation of a machine in an actual field was much higher in points than operating a device after being explained through a PowerPoint presentation.

After the training, one-way ANOVA was performed to determine the difference in understanding of five engine equipment for the experimental group with significantly improved educational effect, and Tukey's test (honestly significant difference) was executed for post-verification. The results of the one-way analysis of variance by the equipment are summarized in Table 6.

As seen from the results of analyzing the survey on the understanding level by equipment, it was confirmed that the value F was 21.778, which was statistically significant (p<0.001). The results of Tukey's post-test analysis revealed that three subsets were formed in the first questionnaire and were marked as a to c. The subsets were formed in the order of E1 (auxiliary boiler) for a, E2 (freezer) for b, E4 (air conditioner), E3 (fresh water generator), and E5 (oil filtration machine) for c. E1 showed the highest understanding level, compared to other subjects, followed by E2. It was observed that the understanding levels of E3 to E5 were the lowest.

Although air conditioners, fresh water generators, and oil filtration devices are installed at a hands-on training center in school for pre-practice to be conducted, main and auxiliary boilers installed on ships are quite large and difficult to install owing to space constraints, unlike small boilers used on land; thus, for most trainees, it was their first-hand experience on such devices.

As in [12], where the effect of the video was high in unfamiliar situations such as cardiac arrest, it is judged that the effect of video education is high in equipment that is encountered for the first time in practical training onboard a vessel.

In terms of trainees' understanding, the post-satisfaction effect of education through videos was significantly higher in the experimental group than in the control group. This was consistent with the study providing the surgical process as video footage [14], and a study verifying the development effect of video education programs for spinal surgery patients [9].

As seen thus far, the reason for video education making a significant difference in the aspect of understanding is that a vessel’s engineers shoot the film on site while operating, and provided an explanation on the devices installed on a ship, so that trainees’ needs were readily met.