The work intensity can be estimated using the heart rate, which was confirmed through previous studies. In addition, when the heart rate increases, energy consumption and skin temperature increase, thereby indicating that the skin temperature changes according to the changes in the heart rate, and the work intensity can be estimated using the heart rate and skin temperature [16][17]. Recently, many studies have been conducted to solve various problems by learning bio signals using deep learning models [18][19]. Using a deep learning-based method, the model can learn the characteristic patterns of various signals to extract representative features from the information of bio-signals with individual differences.

Padraig et al. [20] classified work intensity with a very simple binary classification system using a CNN based on activity data recorded by a smartwatch. The proposed algorithm compared and analyzed the data measured based on Borg's scale with machine learning algorithms such as Support Vector Machine (SVM) and DTW. However, it was difficult to estimate real-time task intensity considering the proposed CNN-based algorithm classifies data by post-processing rather than real-time processing. In particular, the performance of DTW, which is a real-time processing method, was lower.

Darbandy et al. [21] proposed a method for monitoring and detecting occupational physical fatigue using heart rate measurement of a wearable device. It can quickly determine the heart rate signal by determining the similarity using the KNN. However, the proposed method is difficult to apply universally, depending on the environment, given that the user characteristics are not considered. Therefore, it is necessary to study a discrimination algorithm for real-time work intensity based on a user's biological characteristics.

A CNN is a network that can extract spatial features of data, and an LSTM [22] is a network that can learn features of time-series data. CNN-LSTM. These two structures combined can learn the features of time-series data and two-dimensional data together. Because 1D CNN filter 1D data, their features can be obtained from 1D sensor data, and by learning time-series information through LSTM, it shows excellent performance in classifying and estimating time-series data. The 1D CNN-LSTM structure is used in signal-based technologies such as healthcare and RADAR-based Human Activity Recognition (HAR). Shoeibi et al. [23] proposed a model capable of automatically diagnosing schizophrenia by processing an EEG, which is a brain wave signal using a 1D CNN-LSTM structure. This study was conducted to determine whether it is schizophrenia using a 1D CNN-LSTM model by combining graphs according to various electrodes of the EEG signal into one image. The results showed that it is easy to process bio signals by using 1D CNN and LSTM simultaneously. Zhu et al. [24] proposed a model that learns the characteristics of frequency change using 1D CNN-LSTM of the 2D spectrogram of the radar signal in radar-signal-based HAR and classified behavior through LSTMs. The model proposed in this study showed a similar performance to other models using a 2D CNN. Through these studies, the structure of 1D CNN-LSTM can effectively obtain good results in the learning model of time-series data.

In this study, we propose a model for estimating the work intensity of a worker using their heart rate and skin temperature during rest and physical activity states in real time. The proposed deep-learning model comprises a 1D CNN and LSTM, as shown in Figure 1.

Figure 1: 
Structure of proposed work intensity estimation model

The proposed model divides the time-series data of the heart rate and skin temperature in the rest and physical activity states using the sliding window technique for 60 s of the signal, and uses the combined heart rate and skin temperature data as input data. Furthermore, when inputting data into the model, the data is divided into rest and physical activity states such that each state can also be considered. Feature information of time series data are extracted using 1D CNN layers of the preprocessed data in the rest state and physical activity state, respectively. A 1D CNN is a network that can analyze one-dimensional data patterns and time-series information and extracts time-series characteristic information of bio-signals. Then, the extracted feature information is input to LSTM, which can learn time-series data. Here, the initial weights are set by first inputting the characteristic information of the remaining state into the LSTM. Subsequently, the feature information of the physical activity state is input to the LSTM, wherein the weight is mapped with the information of the rest state. Finally, the FCNs were used in a structure for the final output of the data to obtain the estimated work intensity from the output data. By inputting the characteristic rest state first and using the data of the physical activity state later, the characteristic information of the physical activity state can be learned considering the feature information of the rest state. As a result, the work intensity can be estimated considering both states.

The model proposed in this study uses the heart rate and skin temperature data as input data. However, biometric information such as the heart rate and skin temperature differs from person to person depending on the body condition. Therefore, to consider these conditions for all workers, it is necessary to develop a method for identifying variables caused by individual differences. Although there are individual differences for each person, as the work intensity increases, the heart rate and body temperature increase. To apply these human characteristics, this paper uses rest and physical activity, heart rate, and skin temperature data during preprocessing to respond to individual differences in a wide range of work and learn changes in bio signals depending on the work intensity.

The preprocessing process is illustrated in Fig. 2. First, signals in the rest and physical activity states were extracted from the heart rate and skin temperature data. Then, using the Sliding Window technique, the data was overlapped and extracted in units of 60 s. For example, for 180 s of data collected, we obtain 120 pieces of data. However, considering the skin temperature or heart rate does not change rapidly depending on the physical activity state, it is not possible to accurately determine the change if information at a short time point is used. This paper focuses on the fact that the average heart rate is described in terms of heart beats per minute and extracts data in units of 60 s. Finally, to use both the heart rate and skin temperature information simultaneously, the heart rate and skin temperature data at the same time are simultaneously connected and used as input to the model.

Figure 2: 
Method of data preprocessing

The work-intensity estimation algorithm using heart rate and skin temperature uses preprocessed time-series data, as described in Section 3.2. Therefore, the proposed model was designed using a 1D CNN, which can extract of time-series data, and LSTM, which can estimate the time-series data. Previous studies [7]-[13] confirm that heart rate is related to actual work intensity. Moreover, because blood pressure also increases as the heart rate increases, the body temperature can serve as an auxiliary variable for the heart rate. The input time series data was calculated as shown in Figure 3.

Figure 3: 
Process of LSTM in the proposed model

In the 1D CNN, the heart rate at time t (hr_t) and the skin temperature (T_t) are preprocessed input data in 60 s increments are convolved in chronological order to extract temporal characteristics, and the feature data are extracted by combining the other two variables. Then, it goes through multi-layer 1D CNN Layers and max-pooling to generate feature vectors for time-series patterns. For all parameters of the 1D CNN, Kernel Size was set to 2, stride was set to 1, and padding was set to 0. The kernel size of the max-pooling layer was set to 1. The feature vectors in the rest and physical activity states generated through this process were input to the LSTM to extract time-series pattern features. The adapted LSTM is illustrated in Figure 4.

Figure 4: 
Process of 1D CNN Layers

where R_t and E_t are feature vectors extracted from a 1D CNN as input data in the rest and physical activity states, respectively. Each feature vector was processed according to the following formulas.

As shown in Equation (1), R_t is used as an input to LSTM and outputs of Many to One. These vectors are used as the initial weights of LSTM, which E_t will be input in Equation (2), and a vector called output is the output of Many to One. If the output of the vector in the rest state is used as the initial weight of the LSTM, the output considering the characteristics of the rest state can be obtained by inputting the vector in the physical activity state. Therefore, it is possible to obtain a result of estimate the work intensity by considering both the information of the rest and physical activity states. The hidden size of LSTM was set to 128, and the bidirectional option was applied to design a model that simultaneously considers information from the previous and later time steps.

The model proposed in this paper was implemented using Pytorch, a Python-based deep learning framework, and the learning parameters and learning epochs were set to 100. The batch size was set to 512. The learning loss function uses cross-entropy, which is a representative function used in classification algorithms. Adam [25], which optimizes different neural networks well, was used as the learning optimizer, and the learning rate was set to 0.001.

The training dataset used was Fatigueset [26]. Fatigueset measured mental and physical fatigue after performing three types of physical activities by attaching four wearable devices to 12 people. There were three sessions in Fatigueset, as shown in Figure 5.

Figure 5: 
Dataset measurements of Fatigueset

One is S1 Session, wherein the subjects are given a 3 min break for data collection, and the second is S2 Session, wherein the physical activity is performed for 3 min to provide physical fatigue. The third is the S3 Session, which is a task that gives mental fatigue for 15 min. The middle of each course was comprised of M1 sessions that identified fatigue status through questionnaires. The physical activity intensity of the S2 Session was expressed in three categories: 0, 1, and 2, and jogging was performed at speeds of 5, 7, and 9 km/h. When the subject reported a grade of 10, 14, or 16 on Borg’s Scale, the criterion for each physical activity intensity was determined while stopping for safety. We extracted and used the data of sessions S1 and S2 to acquire heart rate and skin temperature information during rest and physical activity states. Heart rate and skin temperature data were extracted from a wrist wearable device, as shown in Figure 6, and signals of 1 Hz and 4 Hz were received. To match the time step of the data, the heart rate and skin temperature data corresponding to 1 Hz were used.

Figure 6: 
Wrist wearable device, E4 Wristband

After completing the preprocessing described in Section 3.2, the total number of data was 466560, and each data point was ([resting heart rate for 60 s, rest skin temperature for 60 s], [physical activity heart rate for 60 s, physical activity skin temperature for 60 s], and physical activity intensity). The ratio of training, verification, and test data was 8:1:1, and bias caused by data order was prevented by randomly shuffling the entire dataset.