Designing container crane control learning model using deep learning

2.1 Container Crane

Container cranes, also known as quay or ship-to-shore cranes, are container-handling equipment employed in loading and unloading operations (vessel operations) when a container vessel docks at a terminal. These cranes facilitate the transfer of containers between the vessel and berth. Their productivity, which is a crucial component of container terminals, is measured as the number of containers handled per hour. These cranes can process between 25 and 30 containers per hour—a metric that significantly influences the overall productivity of a container terminal.

The container crane structure comprises the crane body, which moves along the rails installed on the apron, a folding boom extending from the body toward the vessel, girders extending from the body toward the apron, a trolley for the horizontal movement along the installed rails, a hoist connected to the trolley for the vertical movements of containers, a control room, and devices, such as spreaders, attached to the end of the hoist rope for lifting and lowering containers.

Figure 1 shows the layout of the quay area where vessel operations using container cranes occur. The container crane transfers the containers from the yard to the apron before lifting and moving them toward the vessel. Thereafter, the crane performs loading operations by lowering the containers to specified locations within the vessel or unloading operations by picking up containers from the vessel, moving them toward the apron, and lowering them to their designated positions on the apron or waiting transport vehicles. The container crane operator is responsible for configuring and controlling the container trajectory, managing oscillations and skewness, and preventing collisions between the crane, vessel, and other cranes, thereby ensuring the safe and efficient execution of the operation.

Figure 1:

Vessel operation at quay area

2.2 Dynamic Model of a Container Crane

A dynamic model is employed to simulate container crane movements and transport container oscillations during vessel operations. This model focuses only on the movements of the trolley and hoist, which influence the container loading and unloading operations. The movement of the crane along the rails installed on the apron is not considered, and the oscillations of the container are assumed to occur only in the two-dimensional space formed by the movement of the trolley and vertical line.

Figure 2 shows the plane model of a container crane and its load (container), where x, l, θ are the horizontal position of the trolley, distance from the trolley to the container center, and swing angle, respectively. The load is considered as a point mass, and the mass and stiffness of the hoisting rope and the effects of wind disturbance are neglected. Subsequently, the equation of motion of the container crane system is obtained as follows [5][6]:

Figure 2:

Plane model of a container crane

Equations (1), (2), and (3) represent the dynamics of the trolley, hoist, and container oscillations, respectively. Here, m, m_x, and m_l are the weights of the container, trolley, and hoist, respectively; fx is the force applied to the trolley in the x-direction and f_l denotes the force applied to the hoist in the l-direction; d_vx and dv_l are the viscous damping coefficients associated with the motions along the x - and l-directions, respectively; g denotes the gravitational acceleration.

Based on these equations, the second-order nonlinear coupled differential equations for the dynamic energies of the trolley and hoist are defined as follows:

2.3 Literature Review

Lee [5] proposed a novel dynamic model for overhead cranes in a three-dimensional space based on previous studies on overhead cranes in a two-dimensional space. A two-degree-of-freedom swing angle was defined and used to derive a nonlinear dynamic model of the crane.

Lei et al. [7] designed a dynamic model to capture the threedimensional motion of a spreader based on the movements of the trolley and hoist of a container crane. Using Lagrange’s equations, they obtained a four-degrees-of-freedom dynamic model in the generalized coordinates of the spreader whose motion was analyzed with respect to the distance from the trolley to the container center and its acceleration.

Ismail et al. [3] proposed a sliding-mode control method that considered external interference during maritime container crane operations. LQR was used to determine the sliding surface, and a sliding-mode controller was designed to induce and maintain the state trajectory of the system on the sliding surface. Simulations demonstrated a reduction in the interference effects caused by strong waves and winds during maritime operations.

Lu et al. [4] focused on horizontal motion control and minimized the energy consumption of a double-pendulum crane system. A control problem was formulated based on the dynamic model of a crane system and an energy consumption function was defined. The control problem was discretized using quadratic programming, and the effectiveness of the proposed planning method was demonstrated using hardware experiments.

Zhang et al. [8] designed a PID-based control model for overhead crane position control and sway suppression under uncertain conditions. The parameters were adjusted according to LaSalle's invariance principle to ensure the Lyapunov stability. Experiments with actual cranes validated the robustness of the proposed approach against nonzero conditions and external interference.

Wang et al. [9] developed an intelligent optimization technology based on artificial neural networks for nonlinear overhead crane control. This method eliminated the requirement of an initial stabilization phase and conveniently executed adaptive control based on the developed update rules.

Zhang et al. [10] addressed the overhead crane control problem using an online reinforcement learning algorithm, called passivity-based online deterministic policy gradient. The neural network rules were updated to ensure system stability during learning by analyzing the energy of an overhead crane system.

Zhang et al. [11] addressed the problem of handling various cargo weights in overhead crane control using a deep reinforcement learning algorithm, called domain randomization memory-augmented beta proximal policy optimization (DR-MABPPO). The control problem was formalized as a latent Markov decision process that captured the cargo weight changes as latent variables. Domain randomization enabled the learning algorithm to explore a range of cargo weights. Introducing a beta distribution addressed the action-range issues, ensuring compliance with the physical constraints. Simulations confirmed that DR-MABPPO could learn a versatile control policy for handling cargo of various weights.

Previous research on crane dynamics, control models, and learning models primarily focused on minimizing the sway and energy consumption in overhead crane systems. However, there is a notable gap in the research on learning models for controlling the container trajectories. Therefore, this study designed a deep learning model to predict the trolley and hoist input force values. This enabled the movement of containers from initial positions to target locations, considering diverse cargo weights and trajectories.

3.1 Deep Learning Model

Deep learning or hierarchical learning is defined as a set of machine learning techniques based on artificial neural networks, which combine various nonlinear transformation methods to summarize the crucial content or functionalities within large datasets or complex information. The deep learning model in Figure 3 consists of input and output layers, with hidden layers placed between them. Each layer contains multiple nodes corresponding to an individual perceptron. Nodes within a specific layer are interconnected by those from the preceding and succeeding layers, and the weights represent the activation levels of these connections.

Figure 3:

Basic structure of a deep learning model

RNN is a type of artificial neural network that connects the cells within the hidden layer and sends their outputs to the output layer. These outputs serve as inputs for the next computation within the hidden layer. This recurrent internal structure, which retains the information related to previous time-step results, is useful for learning time-dependent or sequential data. Although RNNs can sequentially process input data of various lengths, they have the limitation of diminishing information transfer from the initial to final stages as the input data length increases. This issue, known as the problem of long-term dependencies in RNNs, leads to information loss as the learning progresses and sequence lengthens.

Figure 4:

Basic structure of an RNN

LSTM, which is a variant of the RNN proposed by Hochreiter et al. [12], was designed to address the long-term dependency problem. The basic operation of LSTM mirrors that of the conventional RNNs; however, the inclusion of the input, forget, and output gates within the cell enables it to resolve long-term memory loss problems. The input gate adjusts the input data, the forget gate determines the influence of the previous cell state using a sigmoid function ranging from 0 to 1, and the output gate modifies the input data by applying weighted values to each state. LSTM addresses the long-term dependency problem by allowing the cell to forget a certain amount of previous cell state data via the forget gate and update the current cell state by multiplying the previous output value and current input value based on the input gate output.

Cho et al. [13][14] proposed GRU, which is a simplified version of LSTM. In GRU, the update gate replaces the forget and input gates of LSTM. A reset gate is included to determine the part of the hidden layer cell, resulting from the previous time step, which is the output.

Figure 5:

Inner structure of individual LSTM and GRU cells

3.2 Model Architecture

The designed deep learning-based container crane control learning model is structured into two phases, as shown in Figure 6.

Figure 6:

Container crane control learning model

3.3 Data Collection

As shown in Figure 7, a container crane simulator was designed to emulate the control method employed by skilled operators to move containers to their target positions. The simulator incorporated the container crane dynamic model described previously and used a real-time PID controller to manipulate the trolley and hoist input force values, thereby replacing the role of the operator in moving the container.

Figure 7:

Container crane simulator

Given the container weight, initial position, and target destination of the container, the simulator generated the input force values to facilitate the container movement based on the following procedure:

2) Output Measurement and Error Calculation

The output values of the inner dynamic model of the controlled object and container crane were measured based on the initial input values. Figure 8 shows the block diagram of the dynamic model of a container crane. The model read f_x andf_l from the input ports and calculated $$ \ddot{x}, \ddot{l}$$, and $$ \ddot{\theta }$$ using the blocks X_2-dot, L_2-dot, and THETA_2-dot representing Equations (4)-(6), respectively. The trolley position, distance from the trolley to the container center, and swing angle of the container were obtained as outputs.

Figure 8:

Block diagram of the container crane dynamic model

Subsequently, the process variables of the horizontal position x of the trolley and distance from the trolley to the container center l were compared with the target values, and the errors were calculated.

3) Calculation of Control Input Values

Using the error values between the process and setpoint variables, the control input values, i.e., f_x and f_l were calculated.

Subsequently, the container crane learning model was trained by generating and collecting data on the target trajectory of the container and the trolley and hoist input force values using the designed container crane simulator. The simulator was configured using dynamic model parameters based on the specifications of an actual container crane. Figure 9 shows that the initial and destination positions were randomly selected within a specified range, considering the size, outreach, and backreach of the container crane. The container weight was also selected randomly within a specified range based on the container weight information loaded onto the container vessels.

Figure 9:

Selection range of the initial and target positions of the container

Table 1:

Selection range of the container weight

In this study, 10,530 cases of data were collected, encompassing different combinations of the initial position, destination, and container weight. These data were used to train the container crane control learning model.

Table 2 lists the specifications of the container crane control learning models based on the designed RNN, LSTM, and GRU. Each learning model comprised a three-layer structure. They were configured as many-to-many models, with the target trajectory of the trolley and hoist and container weight as input features, and the trolley and hoist input force values as output features. Additionally, a rectified linear unit was employed as the activation function for each hidden layer, and adaptive moment estimation served as the optimization function for the learning model.

Table 2:

Detailed configurations of the RNN, LSTM, and GRU models

The collected training data were split into training and validation datasets in the ratio of 80:20, and the training process utilized the mean square error (MSE) as the loss function. The number of epochs and learning rate were set to 5000 and 0.0001, respectively. However, to address the overfitting issues observed during training, the ModelCheckpoint feature was implemented to save the model at an epoch by minimizing the validation loss. These learning models were designed and trained using the Keras package within the TensorFlow library in Python.

4.1 Simulation Environment

To evaluate the performances of the container crane control learning models, three vessel operation scenarios with varying initial positions, target positions, and container weights were defined, as shown in Table 3.

Table 3:

Vessel operation scenarios

Scenario 1 represents the unloading operations, while Scenarios 2 and 3 represent the loading operations. Scenarios 1 and 2 focused on handling empty containers, whereas Scenario 3 involved handling full containers. For each scenario, the input force values of the trolley and hoist predicted by the RNN-, LSTM-, and GRU-based container crane control learning models were compared with the values calculated using the container crane simulator. The normalized root mean square error (NRMSE) was used to evaluate the prediction accuracy. NRMSE is a standardized metric derived by dividing the root mean squared error (RMSE) with the difference between the maximum and minimum values of the actual data. Similar to RMSE, a lower NRMSE value indicates higher predictive accuracy, which has been used in various studies on deep learning prediction models [15][16][17][18].

4.2 Simulation Results

Based on simulation results, the average NRMSE values for the RNN, LSTM, and GRU models were 0.018164, 0.015381, and 0.008362, respectively. All three models demonstrated high prediction accuracy.

• ① In Scenario 1, the NRMSE value of the GRU model was the lowest at 0.006656, followed by those of the LSTM and RNN models at 0.013274 and 0.016513, respectively.
• ② In Scenario 2, the NRMSE value for the GRU model was the lowest at 0.009527, followed by those of the LSTM and RNN models at 0.013172 and 0.02076, respectively.
• ③ In Scenario 3, the NRMSE value of the GRU model was the lowest at 0.02076, followed by those of the RNN and LSTM models at 0.01722 and 0.019699, respectively.

Table 4:

Prediction results of the RNN, LSTM, GRU models

The GRU model consistently outperformed in all the scenarios, confirming its effectiveness in predicting the input force values of the trolley and hoist to move the containers of various weights to the target position, as shown in Figure 10. Figure 11 compares the prediction results of the GRU model and simulator results for each scenario. GRU resolved the issue of long-term dependencies while maintaining a more compact structure, thereby accelerating training and prediction. This can be attributed to the streamlined architecture of the model, which is less sensitive to unnecessary details and facilitates the learning of more general patterns.

Figure 10:

Comparison of prediction results

Figure 11:

Comparison of the GRU model and simulator results

Furthermore, dynamic simulations were conducted for each scenario to assess the control performance of the GRU container crane control learning model. Particularly, the trolley and hoist input force values predicted by the GRU model were used as the input values for the container crane dynamic model, and it was verified whether the container moved to the intended target position.

Figure 12 shows the trolley and hoist trajectories in each scenario, where the input force values predicted by the GRU model were the dynamic model input values. As shown in Table 5, the dynamic simulation results of the GRU container crane control learning model indicated that the final x-axis position error of the container was within 0.1 m for Scenarios 1 and 3 and 0.2 m for Scenario 2. Additionally, the y-axis position error was within 0.1 m for Scenarios 1 and 2 and 0.2 m for Scenario 3.

Figure 12:

Trolley and hoist trajectory in each scenario

Table 5:

Dynamic simulation results of the GRU model

Author Contributions

Conceptualization, H. S. Shin and H. S. Kim; Methodology, H. S. Shin; Software, H. S. Shin; Validation, Y. S. Ha; Formal Analysis, H. S. Shin & Y. S. Ha; Investigation, H. S. Shin & S. P. Lee; Resources, H. S. Shin; Data Curation, H. S. Shin & S. P. Lee; Writing-Original Draft Preparation, H. S. Shin & S. P. Lee; Writing- Review & Editing, Y. S. Ha; Visualization, H. S. Shin; Supervision, H. S. Kim; Project Administration, H. S. Kim; Funding Acquisition, H. S. Kim.