1. Introduction
With the global aging process, there is an increasing number of patients with movement disorders due to natural diseases, accidents, etc. For most patients with brain damage, brain affections often result in motor function impairments, leading to disrupted neuromuscular pathways and subsequent motor dysfunction [Reference Li, Huang, Lin, Zhao, Lo, Mao, Chen, Zhang, Huang and Li1, Reference Laaksonen, Kirveskari, Mäkelä, Kaste, Mustanoja, Nummenmaa, Tatlisumak and Forss2]. These patients are very eager to be able to stand and walk independently. In addition to the necessary medication, rehabilitation training is also very helpful for the recovery of their movement ability. Sit-to-stand exercises are effective in activating the patient’s respiratory and circulatory systems, relieving spasticity, and improving and maintaining lower extremity muscles, which is an important goal in improving the mobility of limbs [Reference Cheng, Chen, Wang and Hong3]. Moreover, the use of quantitative and instrumental neuromotor assessments can foster the design of effective therapeutic interventions and promote the development of personalized therapies to maximize motor recovery effects [Reference Maggioni, Melendez-Calderon, Van Asseldonk, Klamroth-Marganska, Lünenburger, Riener and Van Der Kooij4]. The implementation of rehabilitation robots can provide effective and repetitive training for patients, increasing the opportunity for recovery and reducing the cost of clinical care [Reference Zhao, Zhang, Guan, Deng and Chen5].
With the development of robotics, rehabilitation robots have made great strides in the past few years and are widely used in clinical rehabilitation [Reference Kapsalyamov, Hussain, Goecke, Brown and Jamwal6, Reference Cao, Li, Li, Li and Wang7]. Samsung’s GEMS series of exoskeletons, especially the GEMS-H version with integrated smartphone app control, is suitable for intelligent rehabilitation training [Reference Shin, Byeon, Yu, Yun, Kim, Lim, Kim, Lee and Lee8]. The study by Wang et al. [Reference Wendong, Hanhao, Menghan, Yang, Xiaoqing, Xing and Bing9] presents a flexible exoskeleton for upper limb rehabilitation with a bionic style. The Hybrid Assistive Limb lower limb model developed by CYBERDYNE is a wearable robot that supports the movement of hemiplegic patients by detecting bioelectric signals [Reference Takahashi, Mizukami, Watanabe, Kuroda, Shimizu, Nakajima, Mutsuzaki, Kamada, Tokeji, Hada, Koseki, Yoshikawa, Nakayama, Iwasaki, Kawamoto, Sankai, Yamazaki, Matsumura and Marushima10]. Techno Concept’s Vibramoov PRO provides functional proprioceptive stimulation that induces motor sensation to help bedridden patients improve their gait and postural control [Reference Bhardwaj, Khan and Muzammil11]. Hocoma’s Lokomat is a gait-training robot with body weight support and virtual reality gaming, helping patients achieve a natural gait [Reference Wu, Xu and Wu12]. In addition, MediTouch’s 3DTutor implements wearable motion feedback systems and develops rehabilitation software to support intensive exercises for lower extremity recovery [Reference Lee, Kim, Lim, Lee and Kim13]. Studies in ref. [Reference Siviy, Baker, Quinlivan, Porciuncula, Swaminathan, Awad and Walsh14–Reference Wu and Chen16] suggest that exoskeleton devices can enhance the performance of accessible users and restore mobility to people with limited mobility. From the work in ref. [Reference Liu, Wang, Cheng, Liao, Zhou and Yang17, Reference Canales-Díaz, Olivares-Valenzuela, Ramírez-Arriagada, Cruz-Montecinos, Vilaró, Torres-Castro and Núñez-Cortés18], it finds that rehabilitation robot-assisted balance training in Parkinson’s disease patients not only improves balance performance but also modulates cortical inhibition and promotes neuroplasticity. The robot-assisted rehabilitation of the wrist in orthopedics in a randomized controlled trial is described in ref. [Reference Albanese, Taglione, Gasparini, Grandi, Pettinelli, Sardelli, Catitti, Sandini, Masia and Zenzeri19, Reference Takahashi, Sato and Namiki20], of which the results are comparable to traditional methods, reducing therapist burden and improving assessment accuracy and patient engagement. For patients, they are more receptive to robots with simple mechanical structures, easily wearable and operative interfaces. For rehabilitation trainers, simple, movable, and lightweight devices can greatly reduce their workload. For patients who need flexible and complex operations such as finger flexion and extension rehabilitation [Reference Ketkar, Wolbrecht, Perry and Farrens21], multi-DOF rehabilitation robots can help them restore joint movements as much as possible. However, for patients with stroke-related motor deficits, it is more important to restore their perception of the limb and the general movement of the limb, and corresponding rehabilitation training needs to be simpler, safer, more durable and repeatable operation of the rehabilitation equipment. Therefore, simpler and more efficient single/few-DOF rehabilitation robots become the suitable solution for such demands [Reference Wang, Wang, Chai, Mo and Wang22, Reference Wang, Wang, Wang, Han and Zhang23]. For these patients who have lost part of their motor ability and still retain some residual muscle strength, a multi-degree-of-freedom rehabilitation robot may make the patient completely passive, causing inefficient training as it cannot fully utilize their residual muscle strength, whereas fewer degrees of freedom can improve the rehabilitation efficiency and stimulate the patient’s intention to restore [Reference Zhang, Guo and Sun24, Reference Lin, Qu, Lin, He, Zhang, Wang, Jiang, Guo and Jia25]. Besides the mechanical design, the interaction issues of the robots are gradually concerned. Traditional robots usually depend on force sensors in human-machine interaction, and the interaction control of force signals is carried out through joysticks, remote control and other means [Reference Hu, Hou, Chen, Zhang and Wang26, Reference Zhao, Sun, Shan, Yang, Yang, Liu, Jiang and Hiroshi27]. However, for patients with brain injury and central nervous system diseases, it is inconvenient to conduct the interactions by hand and limbs directly, thus new human-robot interaction approaches are needed in rehabilitation and assistive applications.
Brain-computer interface (BCI) can decode and convert human brain signals into control signals for external devices without the involvement of the peripheral nervous system or muscle tissues [Reference Mane, Chouhan and Guan28]. BCI techniques have been extensively studied in lower limb rehabilitation [Reference Zhou, Yang and Xue29, Reference Camargo-Vargas, Callejas-Cuervo and Mazzoleni30]. Cui et al. [Reference Cui, Li, Huang, Wu, Fu, Liu, Wan, Wang, Zhang, Qiu, Chen, Yang, Zhu, Li and Chen31] have shown that short-term training with BCI combined with lower limb robotics significantly improves lower limb muscle strength in patients with spinal cord injury (SCI), with some patients recovering from complete to incomplete motor impairment. In addition, the potential of BCI application in sit-to-stand transfer has been demonstrated [Reference Zhu, Maurya, Yi and Dutta32]. Zhu et al. [Reference Zhang, Rong and Luo33] proposed a BCI-enhanced knee exoskeleton control system based on fusing the EEG signals and Inertial Measurement Unit (IMU) data, by which real-time detection of motion intentions and precise control of sit-to-stand transfer movement are achieved, significantly reducing the user’s muscle activation (e.g., 13.2% reduction in rectus femoris and 9.7% reduction in medial femoris). However, research on BCI-based sit-to-stand transfer with single-degree-of-freedom rehabilitation robots is still inadequate, so it is meaningful to design BCI on these robots and validate its performance for patients with stroke-related movement disorders.
Motor imagery (MI) refers to the human brain imagining specific movements of the limb. Nayid et al. illustrate the effectiveness of motor imagery in sit-to-stand transfer [Reference Chaisaen, Autthasan, Mingchinda, Leelaarporn, Kunaseth, Tammajarung, Manoonpong, Mukhopadhyay and Wilaiprasitporn34], and a BCI based on motor imagery can stimulate patients’ active motor intentions and control a rehabilitation robot to achieve a certain level of assistance. Motor imagery involves the human brain imagining a specific movement of a limb, and MI-based BCI stimulates the active motion intention of patients and controls the rehabilitation robot to achieve a certain level of assistance. This method can promote the repair or reconstruction of the damaged motor conduction pathway, wake up the dormant synapses and provide compensatory function. Consequently, rehabilitation therapy can be improved and patients’ limb functions can be restored [Reference Khan, Das, Iversen and Puthusserypady35]. In BCI-based rehabilitation training systems, high-precision motion intention recognition plays a critical role. The event-related desynchronization (ERD) is a technique for detecting and analyzing decreases in the amplitude of cortical potentials for the study of brain activity associated with sensory or cognitive events. However, most studies focused on upper limb MI [Reference Ghani, Sultan, Anwar, Farooq and Khan36–Reference Webb, Xiao, Aschenbrenner, Herrnstadt and Menon38] and lower limb movements primarily centered around ankle and knee joint actions, while studies on sit-to-stand movements are limited because the bidirectional motion has no significant distinctions in ERD [Reference Bulea, Prasad, Kilicarslan and Contreras-Vidal39]. Besides MI-BCI, movement-related cortical potential (MRCP)-based BCIs have also been extensively studied and can decode a variety of functional movements [Reference Jeong, Shim, Kim and Lee40, Reference Liu, Chen, Lee, Chavarriaga, Iwane, Bouri, Pei and d. R. Millán41]. The MRCP is a low frequency (0–5 Hz) negative shift in the EEG signal. MRCP reflects brain electrical activity related to the execution of overt or covert movements. MRCP explicates the cortical processes employed in the planning and preparation of movement. Sit-to-stand transfer requires complicated engagement of cerebral cortex resources related to the trunk and lower limbs, which means that the brain has to coordinate different spatial cortical areas associated with movement in each session. The incorporation of MRCP and ERD is a possible approach to enhance electroencephalography (EEG) decoding performance and has been proven to outperform single-trial classification of movements in the identification of motion intention [Reference Laaksonen, Kirveskari, Mäkelä, Kaste, Mustanoja, Nummenmaa, Tatlisumak and Forss2, Reference Liu, Chen, Lee, Chavarriaga, Iwane, Bouri, Pei and d. R. Millán41]. There is a study that demonstrated the advantages of AO-MI in rehabilitation training [Reference Temporiti, Calcagno, Coelli, Marino, Gatti, Bianchi and Galli42], while another work combined AO and MI in sit-to-stand transfer as part of a BCI system, achieving excellent results [Reference Chaisaen, Autthasan, Mingchinda, Leelaarporn, Kunaseth, Tammajarung, Manoonpong, Mukhopadhyay and Wilaiprasitporn34]. Thus, it is necessary to compare the MRCP in MI and AO-MI and identify which paradigm is more effective in facilitating rehabilitation training for providing guidance for the design of BCI systems.
At present, deep learning models are employed for decoding the EEG signals [Reference Zhang, Yao, Wang, Monaghan, Mcalpine and Zhang43] and have been utilized extensively for a multitude of EEG-related tasks, including motor imagery, epilepsy detection, mental workload, sleep stage scoring and emotion recognition tasks [Reference Craik, He and Contreras-Vidal44]. The most typical deep learning models include Convolutional Neural Network (CNN), Recurrent Neural Network, Long Short-Term Memory Network, etc. Models dedicated to decoding EEG were derived, including EEGNet, EEGInception and EEGITNet. Although deep learning models have EEG decoding to some extent [Reference Sturm, Lapuschkin, Samek and Müller45, Reference Freitas, Inocêncio, Lins, Santos and Benedetti46], their effectiveness is still limited, mainly due to the inherent non-linearity of EEG signals [Reference Torres, Medina-DeVilliers, Clarkson, Lerner and Riccardi47]. This characteristic causes exceptional complexity computation in the processing and analysis of EEG signals that limit the performance of deep learning models in EEG signal decoding. A novel neural network, Kolmogorov–Arnold networks (KANs), was proposed by Ziming Liu et al. [Reference Liu, Wang, Vaidya, Ruehle, Halverson, Soljačić, Hou and Tegmark48]. KANs aim to improve the accuracy of deep learning with non-linear problems. It demonstrated the effectiveness of KANs in time series forecasting and achieved significant results on several evaluation metrics [Reference Han, Zhang, Wu, Zhang and Wu49]. Moreover, the performance of KANs was also proven in EEG applications. KANs could adapt to time series like EEG signals, which improved the accuracy of seizure detection [Reference Contreras, Cui, Yu, Huang, Nikpour and Kavehei50]. The ability of KANs to deal with the non-linearity of EEG signals is determined by the learnable activation functions at the network nodes, which can be dynamically shaped based on training data. This significantly improves the flexibility and accuracy of KANs in EEG classification tasks. Although deep neural networks are capable of learning complex features, those with too many layers generally suffer from gradient vanishing or gradient explosion. To solve this problem, He et al. designed a Residual Unit to connect certain layers in the network by skipping one or two layers directly [Reference He, Zhang, Ren and Sun51]. Residual Networks (ResNet) are formed by stacking Residual Units. For the application of ResNet in EEG signal data classification, it learned complex feature combinations [Reference Tian, Wang, Luo and Zhu52]. To further investigate the advantages of the activation functions on the KANs nodes, it is a possible approach to integrate the output information of ResNet into KANs to enhance its decoding capability in EEG signal processing.
In rehabilitation training, the feedback of the patients is particularly important for evaluating the effectiveness of rehabilitation robots. Scale scores and physiological state analysis are frequently used in studies related to sit-to-stand transfers [Reference Sun, Xiao, Chen, Chen, Lu, Wang, Zheng, Zhang and Xiong53, Reference Renton, Tibbles and Topolovec-Vranic54]. However, although scale scores can provide some clinical guidance, they lack sufficient objectivity and dynamic flexibility, while the sole reliance on EEG analysis has some limitations [Reference Foong, Ang, Quek, Guan, Phua, Kuah, Deshmukh, Yam, Rajeswaran, Tang, Chew and Chua55]. EEG focuses on the electrical activity of the cerebral cortex, but fails to comprehensively reflect the functional changes in muscle movement and nerve conduction results in the musculoskeletal system. In patients with stroke hemiplegia, there is an asymmetry in that the healthy lower limb load is greater than that of the affected side during sit-to-stand transfers, which increases the risk of falls [Reference Cheng, Liaw, Wong, Tang, Lee and Lin56]. Lecours et al. [Reference Lecours, Nadeau, Gravel and Teixera-Salmela57] found that in patients with stroke hemiplegia, after the hips left the seat during sit-to-stand transfers, the trunk flexed laterally toward the healthy side, accompanied by an increase in the healthy lower limb weight-bearing. Since the ability of motor control of both lower limbs is basically symmetrical in healthy individuals, there was no significant asymmetry in the weight-bearing of both lower limbs in healthy individuals when completing the sit-to-stand transfer [Reference Özyürek, Demirbüken and Angın58], and the weight-bearing of the dominant leg may be slightly greater than that of the nondominant leg. Lomaglio et al. [Reference Lomaglio and Eng59] concluded that the muscle function of the lower limb on the affected side of hemiplegic patients with stroke and the ability to weight-bearing were important factors in completing the sit-to-stand transfer. Therefore, it is important to evaluate the patient’s rehabilitation from a muscular perspective. More and more studies have started to introduce electromyography (EMG) as a modality of assessment [Reference Chen, Li, Guan, Guo, Wu, Zhou, Huo, Chen and Zhou60, Reference Sevilla and Pinto61]. As an important assessment tool for neuromuscular control, EMG is able to monitor the electrical activity of muscles in real time and can provide direct feedback on muscle responses during exercise [Reference Dal Corso, Nápolis, Malaguti, Gimenes, Albuquerque, Nogueira, De Fuccio, Pereira, Bulle, McFarlane, Nery and Neder62]. Studies have shown that the state of neuromuscular control reflected by EMG signals can be improved by normal perceptual input from brain activity, a process that has a positive effect on neuroplasticity [Reference McFarland and Wolpaw63]. Neuroplasticity is a central mechanism in the rehabilitation process, stimulating the brain and spinal cord to remodel and improve motor function and autonomy during exercise training. For rehabilitation robots, it is clinically meaningful to evaluate their effects in combination with the patient’s neuromuscular condition. By monitoring patients’ EMG activities in real time with the assistance of the rehabilitation robot, the recovery of their muscle strength, coordination and motor control can be objectively assessed. Particularly in patients with lower limb paralysis, the analysis of EMG signals can provide a more accurate and personalized assessment, which can adjust the robotic treatment plan to optimize the rehabilitation outcome [Reference Shiraishi, Kawamoto and Sankai64]. Therefore, there is an urgent need to develop a more efficient, objective and comprehensive assessment method for patients. The integration of EEG and EMG assessment would be beneficial to reduce the limitations of their respective single methods. EEG provides dynamic changes in cortical activity, whereas EMG provides more direct feedback on neuromuscular activity. Combining the analysis of these two types of signals will not only more accurately detect the changes in motor control during sit-to-stand transfer but also better reflect the specific influence of the rehabilitation robot on the patient’s functional recovery. It will also provide an objective and flexible way to evaluate the effectiveness of robotic rehabilitation therapy.
This paper will also analyze and discuss the user experience of the assistive system in addition to the effects of the rehabilitation robot on the human body. Therefore, psychological assessment is also necessary to quantify the level of emotions throughout the rehabilitation process, which can contribute to a more effective follow-up rehabilitation training. In general, mood levels are related to sensations such as comfort in the body [Reference Helander and Zhang65]. A number of studies have shown that among the methods of human emotion recognition involving facial expressions, speech, body movements, EEG and EMG, EEG is more accurate and reliable in digitizing human emotions. During human-machine interaction, participants have corresponding psychological characteristics in the frequency and spatial domains of EEG. The beta-band has a higher correlation with emotions compared to other frequency bands [Reference Yan and Jia66, Reference Fernandes, d. Alexandria, Marques, d. Assis, Motta and d. S. Silva67]. To some extent, it is necessary to measure the emotional state by analyzing the beta frequencies of the EEG for assessing the assist effect of the rehabilitation robot.
The main contributions of this study are as follows:
-
1. A single-degree-of-freedom rehabilitation robot is designed, a BMI system is established for the sit-to-stand rehabilitation training and EEG and EMG coherence is introduced to evaluate the effects of rehabilitation assistance on neuromuscular function.
-
2. Gumbel-ResNet-KANs model is proposed for solving the nonlinearity problems of EEG signals, and its performance in the sit-to-stand motion intention recognition tasks is demonstrated by comparison and ablation experiments.
-
3. This study compares the MRCPs of MI and AO-MI, which lays the foundation for more effective sit-to-stand rehabilitation training and extends the usability of MI-EEG in BMI in lower limbs robotic assistance.
Mechanical structure of the 1-DOF sit-to-stand transfer robot.
2. Methodology
2.1. Mechanical design of the rehabilitation robot
For sit-to-stand rehabilitation, the training process usually involves a large number of repetitive daily rehabilitation movements, which can be quite difficult for most patients [Reference Veerbeek, van Wegen, van Peppen, Van der Wees, Hendriks, Rietberg and Kwakkel68]. Therefore, the structure of a rehabilitation robot should be simple, user-friendly and easy to use and control for both patients and rehabilitation trainers. The 1-DOF lower limb rehabilitation robot is designed based on the method for implementing the task motion by simple planar mechanisms [Reference Li, Lv, Zhao and Lu69] and it needs fewer control commands that facilitate the utilization of BMI. This approach enhances the patient’s proactive engagement in rehabilitation training and improves its effectiveness. In addition, the lower limb rehabilitation robot has the advantages of simplicity, cost-effectiveness and user-friendly interaction, making it suitable for rehabilitation training. To implement rehabilitation training through BMI, the robot is driven by the lifting signal of a push rod motor, which initiates the rotation of the side bar and large gears. The large gears further drive the gear rotation, resulting in the rotation of the rear support bar, ultimately generating the motion trajectory of the seat center to align with the design curve, assisting patients with sit-to-stand or stand-to-sit maneuvers. The design of the sit-to-stand transfer robot’s linkage mechanism is highly adjustable and can be precisely matched to the user’s individual and dynamic needs, thus adapting to the natural curve of movement of different users during sit-to-stand transfers and ensuring smooth and comfortable movements. At the same time, the user is not completely fixed on the robot, allowing the patient to maintain a certain degree of natural movement [Reference Chen, Dong, Lv and Zhu70]. Only one degree of rotational freedom is needed for the safety and efficient repeatability of sit-to-stand training, while the robot’s design prevents injuries caused by unexpected user contact. The overall mechanism and structure are shown in Figure 1. For clear illustration, the seat and a portion of symmetric components have been hidden in the 3D drawing. The rehabilitation robot was designed as a wheelchair that can be raised and lowered through the motion intention recognition results from BMI which was transferred to the controller.
Data acquisition experiment configuration.
2.2. Participants and data acquisition
In this study, 8 healthy participants (male, aged 20–25 years) were recruited. The study involving human participants was reviewed and approved by the Ethics Committee of West China Hospital of Sichuan University. The participants provided their written informed consent to participate in this study. All participants are right-handed, with no history of mental illness or neurological disorders, no use of psychoactive medication, and no cognitive or motor disorders. Before the experiment, participants were informed about the research content and provided oral consent to participate in the experiment. The EEG collection equipment in this study has a 64-channel high-density EEG acquisition module Eego RT from ANT Neuro. The equipment includes the ANT amplifier EE-225, waveguard-shielded EEG cap, eego software and other components. The waveguard-shielded EEG cap is equipped with 64 channels, with electrode positions according to the international 10-20 standard distribution. 32 channels in total were chosen, situated in the frontal lobe, the central area and the parietal lobe close to the motor and premotor cortex including F5, F3, F1, F2, F4, F6, FC5, FC3, FC1, FCz, FC2, FC4, FC6, C5, C3, C1, Cz, C2, C4, C6, CP5, CP3, CP1, CP2, CP4, CP6, P5, P3, P1, P2, P4 and P6 [Reference Al-Saegh, Dawwd and Abdul-Jabbar71]. The surface electromyography (sEMG) data acquisition apparatus utilizes complementary equipment of the Eego RT system. The dual-pole input ports of the eego amplifier EE-225 are compatible with the dual-pole adapter XS-270, used for sEMG signal collection. The drive and control module includes the STM32 microcontroller and corresponding control board, motor driver, etc., integrated into a panel. EEG and EMG signals were synchronously collected at a sampling frequency of 1024 Hz. The data acquisition experiment configuration is illustrated in Figure 2. EEG signals are used in this study as the basis for recognizing the motion intention in the BMI-based rehabilitation robot system, which allows the robot to assist in sit-to-stand transfers for rehabilitation training. The EMG signals generated during human movement reflect the state and health of the limbs and are a commonly used electrophysiological method in limb rehabilitation effect assessment. EEG and EMG signals were acquired synchronously during the experiments, and their coherence was analyzed to provide a comprehensive understanding of the effectiveness of rehabilitation training from both neurological and muscular perspectives.
2.3. Experiment paradigm
For the sit-to-stand rehabilitation training, this study proposed a BMI system for the lower limb assistance robot. The system performed pattern recognition on MRCP in MI and converted them into recognizable control commands that guided specific movements executed by the rehabilitation device, which allowed the brain to control the lower limb rehabilitation device, and the sEMG signals were used to evaluate the effectiveness of the rehabilitation training, to formulate the rehabilitation training. Figure 3 illustrates the sit-to-stand rehabilitation training and evaluation system. The experiment paradigm design depended on the practical needs of the sit-to-stand movement. Participants completed a total of 3 sets of experiments, each with 80 tasks. The first group of sit-to-stand MI sessions followed the general process [Reference Triana-Guzman, Orjuela-Cañon, Jutinico, Mendoza-Montoya and Antelis72]: a 3-second black screen was presented initially, allowing participants to rest. Then, a crosshair appeared on the screen for 2 s, prompting participants to focus their attention. Subsequently, participants started to imagine when an arrow is displayed on the screen and lasts for 4 s. The second group of AO-MI experiments was accompanied by motor observation to overlay the imagery to guide the participants to perform better MI and generate qualified MRCPs in MI [Reference Rungsirisilp and Wongsawat73]. The videos shown in the task were those in which the character performs a complete movement from sitting to standing or from standing to sitting. Furthermore, the cue-based motor execution (ME) was conducted in the third group of experiments. In this experiment, participants engaged in ME corresponding to the displayed action as the reference of original MRCPs. In this experiment, the patients were expected to go through the pre-training of MI and action observation of AO-MI first and then generate MRCPs directly in ME for the comparison of MRCPs from different sources. The workflow of the system is demonstrated in Figure 3.
The workflow of the sit-to-stand rehabilitation training and evaluation system.
2.4. Data processing
In this study, MATLAB’s letswave7 toolbox was used for the preprocessing of EEG signals, extracting both time-frequency MI features and MRCP related ones. The MI data were filtered by a Butterworth filter with a frequency domain range of 0.01-30 Hz, down-sampled to 512 Hz. Independent Component Analysis was employed to identify artifact components within the EEG data, and then the data were low-pass filtered at 3 Hz with a Butterworth filter to obtain the MRCP data.
For EMG signals, baseline correction is necessary because EMG acquisition is wired. A 50 Hz sine wave and its spikes of all harmonics will be present in the sEMG signal and should be removed for EMG analysis. The band-pass filtering is required to guarantee a valid range of 20–500 Hz. The envelope is a continuous curve formed after filtering and leveling rectification of the original EMG signal, which serves for qualitative analysis of EMG characteristics. The envelope of the signal can be extracted by low-pass filtering.
2.5. The construction of EEG decoding model
2.5.1. KolmogorovArnold networks (KANs)
The KANs present a neural network architecture that leverages learnable spline-based functions to enhance the approximation of complex nonlinear relationships. KANs rely on the Kolmogorov–Arnold representation theorem [Reference Liu, Wang, Vaidya, Ruehle, Halverson, Soljačić, Hou and Tegmark48], and it theoretically states that any multivariate continuous function
$f$
, which depends on
$x=[x_1,x_2,\ldots ,x_n]$
, on a bounded domain, can be represented as a composition of univariate functions and the addition operation:
\begin{equation} f\!\left (x_{1}, \ldots , x_{n}\right )=\sum _{q=1}^{2 n+1}\left (\sum _{p=1}^{n} \phi _{q, p}\left (x_{p}\right )\right ) \end{equation}
where
$\phi _{q, p}$
indicates univariate functions that map each input variable
$x_p$
, and
$\phi _{q}$
indicates continuous functions.
In original MLPs, which are based on the universal approximation theorem, weight parameters are typically assigned to the edges of the network, while neurons are described as predefined activation functions. Unlike the ordinary approach, KANs incorporate learnable activation functions on the edges (i.e. “weights”) between nodes. These activation functions adapt dynamically during network training, effectively replacing the network weight parameters with univariate spline functions. This innovative approach allows the network to maintain flexibility while achieving precise fitting of intricate network structures.
A KAN layer is defined by a matrix composed of univariate functions
$\left \{\phi _{q, p}(\cdot )\right \}$
with
$p=1,..,N_{in}$
and
$q=1,\ldots ,N_{out}$
, where
$N_{in}$
and
$N_{out}$
denote the number of inputs and outputs, respectively, and
$\phi _{q, p}$
indicates the trainable spline functions described above. This architectural innovation allows KANs to better capture complex, nonlinear relationships more effectively than traditional MLPs. Note that in the Kolmogorov–Arnold theorem, the inner functions constitute a KAN layer with
$N_{in}=n$
and
$N_{out}=2n+1$
, while the external functions formulate a KAN layer with
$N_{in}=2n+1$
and
$N_{out}=1$
.To extend the capabilities of KANs, deeper network architectures have been developed. A deeper KAN is essentially a composition of multiple KAN layers, with an enhanced ability to model more complex functions. A general KAN can be expressed by the composition of layers [Reference Liu, Wang, Vaidya, Ruehle, Halverson, Soljačić, Hou and Tegmark48]:
\begin{equation} y=K A N(x)=\left (\Phi _{L} \circ \Phi _{L-1} \circ \cdots \circ \Phi _{1} \circ \Phi _{0}\right ) \end{equation}
In the implementation of KAN and KANLinear, we carefully selected the hyperparameters to ensure that the models have good fitting ability and generalization performance when dealing with complex data. For the KAN layer, we designed the structure with a width of 2 neurons in the input layer, 5 neurons in the hidden layer and 1 neuron in the output layer. This design aims to balance the expressiveness and complexity of the model and ensure the generalization ability of the model on complex data by reasonably controlling the width of the network while avoiding the overfitting problem caused by too wide a network. The number of input features of the KANLinear layer is dynamically determined, depending on the output of the previous layer. This design allows the model to flexibly adapt to different stages of feature extraction requirements, while its output feature number is also dynamically adjusted according to the output dimension of the specific task, thus ensuring the adaptability and robustness of the model in different tasks. In the selection of the spline function, we choose 5 grid sizes to ensure that the spline function can effectively capture the local features of the data. In addition, we adopted a 3rd-order spline because it achieves a good balance between smoothness and fitting ability, which enhances the model’s adaptability to complex data. The parameter selection is detailed and described in Section 3.2.1.
2.5.2. The setting of ResNet34.
During the training of deep learning networks, gradients vanish or explode usually occurs. To solve these problems, He et al. proposed ResNet, an innovative network architecture. The main advantage of ResNet is that it can effectively mitigate the gradient vanishing problem in deep learning, thus improving the training efficiency and performance of deep networks. Through its unique residual learning mechanism, ResNet is able to maintain the depth of the network while ensuring the effective transfer of information and gradients, which is crucial for building high-performance deep learning networks [Reference He, Zhang, Ren and Sun51]. In this study, ResNet34 was applied, and Table I shows the specific parameters of ResNet34.
ResNet34 architecture.
2.5.3. Gumbel-softmax
The input layer takes EEG data with extracted MRCP features after preprocessing and feeds it to the Gumbel-Softmax layer for channel selection. For the input EEG signals, requiring the selection of M channels, a probability matrix
$p_{cm}=[p_{c1},p_{c2},\ldots ,p_{cm}]$
for candidate channels is constructed. Discretization of the candidate channel selection is achieved through the following calculation:
\begin{equation} h=o n e \_h o t[\arg \max(\log P_{c i}+g_{i})] \end{equation}
where
$g_i$
represents Gumbel noise obtained by applying the log function twice to samples from a uniform distribution. During training, the derivative of one-hot is approximated by the continuous and differentiable Gumbel-Softmax function:
\begin{equation} h_{i}=\frac {\exp\! \left (\left (\log \pi _{i}+g_{i}\right ) / \tau \right )}{\sum _{i=1}^{m} \exp \left (\left (\log \pi _{i}+g_{i}\right ) / \tau \right )} \end{equation}
Where
$\tau$
is the annealing parameter controlling the degree of similarity between the sampled result and one-hot encoding.
Structure of the Gumbel-ResNet-KANs model.
2.5.4. Gumbel-ResNet-KANs
The KANs is combined with the ResNet-34 network to form ResNet-KANs. MRCP performs channel selection through the Gumbel-softmax layer before inputting ResNet-KANs. This integration prevents the increase in the nonlinear structure of the network, enhances the representation of the model, and improves the accuracy of the classification results. Compared with the original ResNet network, Gumble-ResNet-KANs not only improve the learning ability of the network, capture and utilize the complex structural information in the data more effectively but also adapt to the non-linearity of the EEG signal. Figure 4 shows the model constructed in this study. The preprocessed MRCP data were first fed into a Gumbel-Softmax layer to select the desired channels. Subsequently, a residual block containing two 3x3 convolutional layers, each followed by a batch normalization layer and a ReLU activation function, was defined to build Layer1-4. Finally, the output of global average pooling is conveyed to KANs for classifying the MRCPs generated in the sit-to-stand transfer classification task.
2.6. Assist effect evaluation
Assist effect evaluation is the process of assessing the assistance outcome of patients undergoing rehabilitation training or therapy according to certain standards, which is one of the most important steps of the patient’s rehabilitation process. There are various assessment methods for human-robot interaction [Reference Das and Mukhopadhyay74]. For better sit-to-stand transfer rehabilitation by robots, it is also meaningful to evaluate the assist effect from neuromuscular perspectives. EEG and EMG coherence analysis can represent the oscillatory relationship between the brain and the peripheral neuromuscular system [Reference Youssofzadeh, Zanotto, Wong-Lin, Agrawal and Prasad75], which in turn assesses the functional state of the neuromotor system [Reference Xie, Zhou, Chen, Yang, Zhang and Hu76] and the functional changes in the neuromotor system assisted by the rehabilitation robot. This experiment collected EEG and sEMG data simultaneously and analyzed the relationship between EEG and sEMG coherence with assisted standing and without it. The wavelet coherence was analyzed, as it could retain the time-domain coherence information between two signals. After preprocessing the EEG and sEMG data, the Cz channel and bilateral rectus femoris EMG signals were selected for coherence analysis. The wavelet coherence between two time series is defined as:
\begin{equation} M = \frac { | S ( C _ { x } ^ { * } ( a , b ) C _ { y } ( a , b ) ) | ^ { 2 } } { S ( | C _ { x } ( a , b ) | ^ { 2 } ) \cdot S ( | C _ { y } ( a , b ) | ^ { 2 } ) } \end{equation}
where
$ C _ { x } ( a , b )$
and
$ C _ { y } ( a , b )$
represent the CWT of
$x$
and
$y$
on scale
$a$
and position
$b$
, the
$*$
denotes the complex conjugate and
$S$
is a smoothing operator over time and scale.
Similar to the EEG-EMG coherence, emotional state in human-robot interaction could also be measured by EEG. Due to the variations of rehabilitation motivation among participants, the beta waves of EEG were evaluated for statistical analysis. Difference Entropy (DE), Difference Asymmetry (DASM) and Rational Asymmetry (RASM) were used to objectively assess the participants’ mood levels, which may reflect the participants’ level of activeness and comfort. These indices have been proven effective in evaluating emotional valence [Reference Zheng, Zhu and Lu77].
DE extends the concept of discrete Shannon entropy to measure the complexity of continuous random variables. It is often used for emotion recognition and has shown good performance. DE is defined as:
\begin{equation} DE(X) = -\int _{-\infty }^{\infty } \frac {1}{\sqrt {2\pi \sigma ^2}} e^{-\frac {(x-\mu )^2}{2\sigma ^2}} \log \left ( \frac {1}{\sqrt {2\pi \sigma ^2}} e^{-\frac {(x-\mu )^2}{2\sigma ^2}} \right ) dx = \frac {1}{2} \log (2\pi e \sigma ^2) \end{equation}
Where the time series
$X$
obeys a Gaussian distribution
$N(\mu , \sigma ^2)$
.
Essentially, EEG-based asymmetry is helpful for negative emotion recognition [Reference Harmon-Jones and Allen78]. Asymmetry occurs in the left-right and anterior-posterior hemispheres of the brain [Reference Davidson79]. The EEG cap has 28 pairs of electrodes in the left and right hemispheres and 27 pairs in the anterior and posterior hemispheres, respectively. DASM and RASM features are defined as the differences and ratios between the DE features of 28 pairs of hemisphere-asymmetric electrodes [Reference Fernandes, d. Alexandria, Marques, d. Assis, Motta and d. S. Silva67, Reference Liu, Lv, Shen, Xiong, Yang and Liu80].
\begin{equation} DASM = DE(X_{left}) - DE(X_{right}) \end{equation}
\begin{equation} RASM = \frac {DE(X_{left})}{DE(X_{right})} \end{equation}
Where
$X_{left}$
and
$X_{right}$
denote electrode pairs in the left and right brain hemispheres.
3. Results
3.1. Validation of MRCPs for classification
Event-related desynchronization/synchronization (ERD/ERS) phenomena are an important indicator of the rhythmic characteristics in MI EEG signals. Figure 5 illustrates the FCz and Cz channels data of the time-frequency spectrograms by the superposition and average for participant S01 during MI and AO-MI tasks. Panels (a) and (b) correspond to original MI tasks with arrow prompts, while panels (c) and (d) represent AO-MI tasks. The color bar on the right indicates energy levels, with time on the horizontal axis and frequency on the vertical axis. The period from −1 s to 0 s represents the task preparation phase, during which participants focus their attention, followed by task execution from 0s. The figures show energy decreases in the theta (4–8 Hz) and alpha (8–13 Hz) bands during task execution, corresponding to ERD, which suggests the influence of sit-to-stand MI tasks mainly in the theta and alpha frequency ranges of the cerebral cortex. Additionally, AO-MI tasks appear obvious ERD compared to traditional MI tasks, indicating that action observation enhances the effect of MI.
The time-frequency spectrograms for sts, (a, b) for motor imagery, (c, d) for action observation of motor imagery.
Movement-related cortical potentials (MRCP) during the execution of different direction sit-to-stand transfers: (a) for MRCP in the FCz channel and (b) for MRCP in the Cz channel.
This study focuses on analyzing variations in MRCP in MI during the preparation for sit-to-stand transition movements. MRCP is closely related to MI and ME in multiple motor-related functional areas of the cerebral cortex. The MRCPs from different trials of each action were averaged, and FCz and Cz channels on the region of the central sulcus gyrus on the sensorimotor cortex were selected for analysis. Figure 6 shows the data of participant S02: the X-axis represents the time from −2 s to 3 s, and 0 s is the time of marking. (a) represents MRCP in the FCz channel, and (b) represents MRCP in the Cz channel. The red curve corresponds to the sit-to-stand action while the blue one to the stand-to-sit action. Both actions have strong negative deflection, with a reduction of around 500 ms before the movement starts, reaching maximum amplitude when the movement begins. Both FCz and Cz channels show that the MRCP in MI for sit-to-stand is larger than those for stand-to-sit, indicating stronger neural activity in the cortex for the sit-to-stand motion. The MRCP feature differences between the two actions provide an alternative approach for motor rehabilitation by the robot with a BMI module.
Figure 7 compares the differences in MRCPs in the Cz channel during sit-to-stand and stand-to-sit actions in both AO-MI and ME modes of participant S02. AO-MI instead of MI was used for comparison because of its better performance. It can be observed that a negative shift in the low-frequency band of the EEG signal can also be seen before the MI of the performed action. Regardless of sit-to-stand or stand-to-sit, the MRCP amplitudes in AO-MI are smaller than in actual ME, suggesting that similar brain activation patterns are generated in the two types of paradigms only with different brain activation levels. Therefore, MRCPs in MI can be considered as another electrophysiology characteristic for bidirectional motion intention recognition in MI-based BMI and MRCPs in AO-MI are more suitable for their intensity.
Comparing movement-related cortical potentials in both action observation of motor imagery and motor execution: (a) for sit-to-stand and (b) for stand-to-sit.
3.2. Decoding results of EEG signals
3.2.1. Training procedure
Hyperparameter variation and model accuracy: (a) spline function order, (b) grid size, (c) learning rate and (d) weights decay.
We made several optimal choices of parameters in the network to improve the accuracy of the network and avoid the overfitting phenomenon. In Figure 8, this paper shows the effect of variation of spline function order and grid size on model accuracy in KAN. The results show that the accuracy of both AOMI and MI tasks reaches the highest value when the spline function is in the 3rd order. When the spline function order is raised to 5, the average accuracy of the AOMI task is lower than that of the 3rd order, although the maximum accuracy of the AOMI task reaches 100%. This suggests that too high an order of the spline function may lead to overfitting phenomena, while too low an order may limit the model’s ability to learn complex features. In contrast, the change in grid size had less impact on the accuracy of both types of tasks. In addition, this paper analyses the effect of learning rate and decay weights on the accuracy of the two types of tasks. As can be seen from Figure 8, changes in the learning rate have a large impact on the model accuracy, and the model performs best when the learning rate is set to 0.001. A learning rate that is too high (e.g. 0.01) will result in the model failing to converge, while a learning rate that is too low may cause the model to fall into overfitting. The effect of decay weights on the model is thought to be much smaller than the learning rate, and for decay weights, 0.01 is the best choice. Too low a decay weight may lead to model overfitting, while too high a decay weight may make the model underfitting. In summary, the accuracy and generalization ability of the model can be effectively improved by reasonably choosing the spline function order, grid size, learning rate and decay weights.
Loss and accuracy during training of the Gumbel-RseNet-KANs.
A batch size of 16 was selected to enable efficient training. We used AdamW optimizer to prevent overfitting. We set the initial learning rate at 0.001 and applied a weight decay value of 0.01 to prevent overfitting. We adopted the CrossEntropyLoss function as the main loss criterion. Each model undergoes 500 epochs of training with the above settings, and the highest accuracy achieved on the test set is regarded as the model’s final performance. Figure 9 shows the loss function and accuracy of the training process. The model exhibits rapid convergence during the first 200 epochs and stabilizes after 300 epochs, then the curve becomes smoother beyond 400 epochs.
3.2.2. Decoding results of the Gumbel-ResNet-KANs
The problem of most BMI systems based on MI sensorimotor rhythmic features is that they require multiple training sessions until the user can adequately control themselves to generate such signals. By contrast, MRCP in MI is easily generated by inexperienced BMI participants without the requirement for extensive pre-training and can be observed accompanied with MI. To classify MRCP of sit-to-stand in different trials, the Gumbel-ResNet-KANs model is introduced for motion intention recognition. The best accuracy of MRCP in MI and AO-MI for all participants is presented in Table II, the accuracies of the classification are better for those in AO-MI, which indicates that the MRCPs in AO-MI of different actions can be better identified and could be regarded as typical characteristics in MI-EEG. Wilcoxon test is a suitable non-parametric test due to the small sample size and paired data of this study [Reference Gehan81]. In this paper, a Wilcoxon test of the results yields p = 0.0412, proving that there is a significant difference in the performance of the model on the two datasets. It also reflects that MRCPs in AO-MI are similar to original MRCPs of ME in motion intention recognition for sit-to-stand transfer interaction.
Accuracy of Gumbel-ResNet-KANs in Motor imagery-movement-related cortical potential (MI-MRCP) and Action observation of motor imagery-movement-related cortical potential (AO-MI-MRCP)
3.2.3. Comparative experiment
To further evaluate the performance of the Gumbel-ResNet-KANs network, several networks that are widely applied in the field of EEG were introduced in this study for comparison. Specifically, seven networks, EEGNet, EEGITNet, EEGResNet, EEGInception, Shallow ConvNet, Deep ConvNet and EEGComform, are selected in this study. Among them, EEGNet is a compact CNN designed for EEG signals that use deep separable convolution to extract features for efficient feature learning and good generalization capability; EEGITNet combines time-frequency analysis and deep learning by extracting time-frequency features through a short-time Fourier transform, and then classifying them using a convolutional network in order to enhance the capture of time-frequency characteristics of EEG signals; EEGResNet adopted a residual network architecture to solve the deep network training problem through residual connectivity, enabling the network to learn complex feature representations of EEG signals and EEGInception, constructed by the Inception module, enhances the ability to capture multi-scale features of EEG signals through parallel multi-scale convolutional kernels for feature extraction. Shallow ConvNet extracts band power features efficiently through temporal convolution and spatial filtering, and has good decoding performance, while Deep ConvNet extracts complex features of EEG signals step by step through multi-layer convolution and pooling operations, which is suitable for processing complex EEG data. EEGConformer combines the advantages of convolutional neural networks and Transformer to improve the decoding ability by extracting local features through the convolutional module and capturing global temporal dependencies through the self-attentive module [Reference Altaheri, Muhammad, Alsulaiman, Amin, Altuwaijri, Abdul, Bencherif and Faisal82]. These networks optimize the processing and classification of EEG signals through their respective strategies. In this study, their performances on the classification tasks of MRCPs in AO-MI and MI for eight subjects were compared and analyzed. The results of the AO-MI experiments are shown in Table III and Figure 10. The results of the MI experiments are shown in Table IV and Figure 11.
Action observation of motor imagery-movement-related cortical potential comparative experiment accuracy. The models are named as follows: Model1 (Shallow ConvNet), Model2 (EEGIception), Model3 (EEGITNet), Model4 (EEGResNet), Model5 (Deep ConvNet), Model6 (EEGNet), Model7 (EEGConformer), and Model8 (Gumbel-ResNet-KANs).
Action observation of motor imagery-movement-related cortical potential comparative experiment accuracy (a) Comparison of accuracy for each model. (b) Comparison of average accuracy and standard deviation on each model.
The comparative experiment demonstrates that the Gumbel-ResNet-KANs performs best in multiple subjects. The mean accuracy of the Gumbel-ResNet-KANs model is higher than that of the other models. In the AO-MI-MRCP classification task, the average accuracy of the Gumbel-ResNet-KANs is 85.94% (see Table III). This represents a 7.03% improvement over the performance of EEGNet and EEGcomform, which have the highest accuracy among the other seven networks. In the MI classification task, the Gumbel-ResNet-KANs achieve an average accuracy of 78.91%(see Table IV). This is 3.91% more accurate than Deep ConvNet, the most accurate of the seven networks. In participants S01 and S02, the classification accuracy of Gumbel-ResNet-KANs in AO-MI-MRCP reaches 93.75%, which is much higher than the other networks. This tendency was also demonstrated in other participants, proving the effectiveness of the Gumbel-ResNet-KANs in dealing with complex EEG signals. As above, we statistically analyzed the performance of eight models on two datasets, AO-MI-MRCP and MI-MRCP to demonstrate the superiority of the proposed model. In this paper, we statistically analyzed the performance of eight models on two datasets, AO-MI-MRCP and MI-MRCP. The Friedman test showed that there were significant differences in the performance between the models in both datasets. To compare results in multiple experimental conditions, the Friedman-Nemenyi test is a suitable nonparametric method [Reference Herbold83]. Further Nemenyi test shows that on the AO-MI-MRCP dataset, the Gumbel-ResNet-KANs model proposed in this paper is significantly different (
$p\lt 0.05$
, see Table V) from all the comparison models, demonstrating a clear performance advantage; while on the MI-MRCP dataset, the model also compares well with EEGIception, EEGITNet and EEGNet and other mainstream methods with significant advantages (
$p\lt 0.05$
, see Table VI). These experimental results fully validate the effectiveness and advantage of the Gumbel-ResNet-KANs model, indicating its better performance and stronger competitiveness in EEG signal classification tasks.
Motor imagery-movement-related cortical potential comparative experiment accuracy.
Motor imagery-movement-related cortical potential comparative experiment accuracy (a) Comparison of accuracy for each model. (b) Comparison of average accuracy and standard deviation on each model.
Gumbel-ResNet-KANs show a stronger capability in the recognition of nonlinear features of EEG, this is due to the fact that compared with models such as EEGNet, the model constructed in this study can both learn complex feature combinations with ResNet and adapt to the non-linearity of EEG signals with activation functions on KANs nodes. These results show that the Gumbel-ResNet-KANs has obvious advantages in EEG.
In this study, it can also be observed that the accuracy in AO-MI-MRCP is higher than MI-MRCP in the prevalent classification networks for EEG, and it provides evidence that MRCPs in AO-MI can be better recognized and can be used as typical characteristics.
3.2.4. Ablation experiment
To illustrate the contributions of ResNet34 and KANs, this study designed ablation experiments by removing the ResNet34 and KAN modules, respectively, in order to observe their specific effects on network performance. The results of the AO-MI experiments are shown in Table VII and Figure 12. The results of the MI experiments are shown in Table VIII and Figure 13.
Action observation of motor imagery-movement-related cortical potential comparative experiment Friedman-Nemenyi test result, critical value = 2.4535.
Motor imagery-movement-related cortical potential comparative experiment Friedman-Nemenyi test result,critical value = 2.4535.
Action observation of motor imagery-movement-related cortical potential ablation experiment accuracy.
Action observation of motor imagery-movement-related cortical potential ablation experiment accuracy. (a) Ablation of accuracy for each model. (b) Ablation of average accuracy and standard deviation on each model.
In the ablation experiments, Gumbel-KANs show similar or slightly lower performance than Gumbel-ResNet in some subjects, with an average accuracy of 77.34% in the AO-MI-MRCP classification task, lower than ResNet’s 80.50%, in the MI-MRCP classification task, the accuracy is 72.66%, lower than ResNet’s 74.97%. This is due to the fact that Gumbel-ResNet is able to combine feature information from different layers more efficiently through residual linkage. It enhances the model’s ability to recognize the complex patterns of EEG signals, thus achieving a higher accuracy in the classification task.
The average accuracy of Gumbel-ResNet-KANs exceeds that of Gumbel-ResNet and Gumbel-KANs in the AO-MI-MRCP (85.94%) and MI-MRCP (78.91%) classification tasks. This is due to the fact that ResNet significantly enhances the model’s ability to handle nonlinearities through the introduction of residual linking, which allows the model to learn complex feature combinations. Incorporating the ResNet information output into KANs can more fully exploit the advantages of the activation functions on the KANs nodes, improving the model’s ability to process nonlinear EEG signals and thus obtaining higher classification accuracies. In this paper, the Gumbel-ResNet-KANs model was systematically evaluated by ablation experiments. Friedman’s test on both AO-MI-MRCP and MI-MRCP datasets showed significant differences between the models. Further Nemenyi tests showed that on the AO-MI-MRCP dataset, the full model had a significant advantage over Gumbel-KANs, while the difference with Gumbel-ResNet did not reach the significant level (
$p=0.0628$
, see Table IX) but the average ranking was still 1.000 higher. Similarly, on the MI-MRCP dataset, the full model also significantly outperformed Gumbel-KANs, although the difference with Gumbel-ResNet does not reach a significant level, the ranking difference is 0.875 (see Table X). This trend also proves that KANs can make the model more adaptable. Additionally, it demonstrates the effect of the effective combination of the KANs module with the ResNet architecture on the model performance and shows that the ResNet structure itself has an important contribution to the model performance.
In Table XI, we evaluate the complexity of each model by trainable parameters and floating point operations per second (FLOPs). Trainable parameters are the number of parameters that are tuned by optimization algorithms (e.g. gradient descent) during model training. Typically, the higher the number of trainable parameters, the more complex features the model can learn, but it increases the demand for storage. FLOPs indicate the number of floating point operations required for a single forward or backward propagation, with higher FLOPs implying larger resource consumption [Reference Chen, Kao, He, Zhuo, Wen, Lee and Chan84].
In our proposed Gumbel-ResNet-KANs model, the trainable parameters are 7.537 M and the FLOPs are 1.426 G. The similar Gumble-ResNet model has 7.208 M and 1.425 G. In comparison, the Gumbel-KANs model has only 44.096 K trainable parameters and 2.097 FLOPs. Although Separated ResNet significantly reduces storage and computation requirements, its average accuracy decreases by 8.6% in the AO-MI-MRCP task and 6.25% in the MI-MRCP task, suggesting that ResNet is able to learn more complex information with more parameters. On the other hand, the separation KANs, although the reduction of storage and computation requirements is not significant, the accuracy decreases by 5.44% in the AO-MI-MRCP task and 3.97% in the MI-MRCP task. This trend suggests that the introduction of KANs allows the model to better adapt to the complex features of the EEG by the spline function.
The results of the ablation experiments reveal the respective contributions of ResNet and KANs in the Gumbel-ResNet-KANs network. This combination not only improves the nonlinear modeling ability of the model but also enhances its generalization ability for high-dimensional data, enabling the network to capture and utilize the complex information in the data more efficiently, and thus achieving higher accuracy in the sit-stand transfer classification task.
Motor imagery-movement-related cortical potential ablation experiment accuracy.
Motor imagery-movement-related cortical potential ablation experiment accuracy. (a) Ablation of accuracy for each model. (b) Ablation of average accuracy and standard deviation on each model.
3.3. Assist effect evaluation
3.3.1. Brain-muscle synergy analysis in rehabilitation training based on EEG-EMG coherence
Figure 14 depicts the time-frequency diagrams of the mean values of EEG-EMG coherence during the sit-to-stand process with the assistive rehabilitation robot and without it for participant S04, where (a) and (b) represent the standing process without assistance while (c) and (d) with assistance. Table XII presents the coherence coefficients of Cz channel EEG signals in the alpha and beta bands with the left and right rectus femoris muscles as well as the biceps femoris muscles during the sit-to-stand process with and without the assistive robot. M1-M4 represents the coherence coefficients of Cz EEG with the right and left rectus femoris muscles as well as the right and left biceps femoris muscles, respectively. In the presence of assist, the reduced coherence in the alpha band may be associated with tasks that require a greater allocation of attention and reflect reduced neural activity in the brain during processing cognitive information. In particular, when motor and cognitive tasks need to be processed simultaneously, the reduced coherence in the alpha frequency band suggests that participants may require fewer attentional resources and cognitive loads when performing these tasks, which could help participants alleviate mental fatigue induced by cognitive loads and allow the brain to allocate attentional resources more efficiently [Reference Yamanaka, Horiuchi and Nojima85, Reference Liu, Sheng and Liu86]. Meanwhile, the reduction in coherence in the beta band suggests that assistive devices enhance dynamic adjustment and multisensory integration. This change reflects the brain’s adaptive tuning to the assistive device, which improves motor flexibility and efficiency by reducing coherence in the beta frequency band [Reference Gwin and Ferris87]. This suggests that assistive devices not only reduced the cognitive load on participants but also enhanced adaptation to dynamic tasks. In Table XII, the data in conditions without robotic assistance is larger than assisted ones except for the M3 data of participant S02 while other participants in assistive conditions are greater than those without assistance. Both Figure 14 and Table XII indicate that the coherence coefficients without assistance are greater than those with assistance. Compared to assisted sit-to-stand transfer, completing the task without assistance requires the central nervous system to concentrate on the control of muscle strength, proving the effectiveness of the rehabilitation robot. Figure 15 illustrates the average EEG-EMG coherence coefficients for all participants. Wilcoxon test was performed on the results M1 (
$p=0.0078$
), M2 (
$p=0.0078$
), M3 (
$p=0.1094$
), M4 (
$p=0.0078$
) respectively, except for M3, the rest of the cerebral EMG coherence showed a significant difference between assisted and unassisted ones, which further illustrates the effectiveness of the rehabilitative robotic assistance. During unassisted sit-to-stand transfer, the correlation between the cerebral cortex and muscles is larger compared to assisted standing with the robot. This could be attributed to more requirements of brain and muscle resources during the unassisted process, which is meaningful to assess the effectiveness of the rehabilitation robot and BMI module in the interaction.
Action observation of motor imagery-movement-related cortical potential ablation experiment Friedman-Nemenyi test result,critical value = 1.0600.
Motor imagery-movement-related cortical potential ablation experiment Friedman-Nemenyi test result,critical value = 1.0600.
Number of trainable parameters and FLOPs of the ablation experiment models.
Electroencephalography-electromyography coherence with and without assisted standing process.
3.3.2. EEG-based assessment of emotional level in rehabilitation training in brain-machine interaction
In order to make the rehabilitation robot more relevant to the actual needs of clinical applications, the assessment of the emotional state of the participants is an indispensable part of the process. Positive emotions can increase patients’ confidence and promote motivation engagement and comfort, thereby improving their learning outcomes in the rehabilitation setting [Reference Krasovsky88]. Emotional levels are positively correlated with recovery and survival from physical illness [Reference Lamers, Bolier, Westerhof, Smit and Bohlmeijer89]. This paper will provide an important reference for the development of BCI systems by quantifying the emotional state of users during rehabilitation training. By analyzing the impact of assistive devices on users’ emotions, it helps to optimize the design of assistive technology to ensure that it not only enhances the physical effects of rehabilitation training but also promotes users’ confidence and motivation, thus improving the overall effectiveness of rehabilitation training.
The beta waves of EEG for 8 participants were processed for mood levels. The 64 electrode channels for each participant in the assisted and unassisted sit-to-stand trials were averaged to obtain DASM and RASM for 8 participants, where 1 corresponded to the unassisted trial and 2 to the assisted trial. The mean values of mood levels for all participants are presented in Table XIII. Wilcoxon test for DASM and RASM, respectively, gives DASM (
$p=0.078$
), RASM (
$p=0.0156$
).
Coefficients of coherence of Cz-channel electroencephalography signals in the alpha and beta frequency bands with the left and right rectus femoris and biceps femoris muscles in both conditions with and without assisted standing.
Electroencephalography-electromyography coherence coefficients.
Approach motivation is associated with positive emotions (e.g., happiness) and is commonly associated with enhanced left brain activity; avoidance motivation is associated with negative emotions (e.g., fear) and is commonly associated with enhanced right brain activity, and increased brain asymmetry is also commonly associated with enhanced negative emotions [Reference Harmon-Jones and Allen78]. In the unassisted condition, the mean value of DASM was −12.8921, indicating a large difference in DE between the right and left brain hemispheres. Contrarily, in the assisted condition, the mean value of DASM reaches −9.8475, indicating that the use of assistive devices reduced the DE difference between the right and left brain hemispheres, although the change in DASM did not reach statistical significance this trend suggests that the assistive devices had a positive impact on the participants. For RASM, the mean value for the unassisted condition was 0.9744, while the mean value for the assisted condition was 0.9815, indicating that the use of assistive devices increased the ratio of DE in the right and left brain hemispheres, a change that was statistically significant. Thus the participants’ mood levels improved in the assisted situation, which may have increased the participants’ level of active rehabilitation and comfort [Reference Otaka, Osawa, Kato, Obayashi, Uehara, Kamiya, Mizuno, Hashide and Kondo90]. The rehabilitation robot increased the positive mood of the participants by improving the symmetry of their left and right brains, which provides evidence of the effectiveness of the rehabilitation robot system established in this paper.
4. Discussions
4.1. The necessity of EMG in the BMI system
In the introduction section, it points out that in stroke patients with lower limb hemiplegia, the weight borne by the healthy lower limb is usually larger than that of the affected side when performing sit-to-stand transfers, which suggests the existence of a significant symmetry. To transfer the proposed BMI system to clinical scenario, this paper introduces EMG as part of the assist effect evaluation metrics to simulate such asymmetry of stroke patients. To further discuss the necessity of EMG metrics, and whether EMG signals can be used as an effective evaluation index for stroke patients, this paper will provide evidence through the following experiments and data analyses.
Difference Asymmetry (DASM) and Rational Asymmetry (RASM) in assisted and unassisted situations for all participants, where 1 corresponded to the unassisted trial and 2 to the assisted trial.
Schematic diagram of the experiment to simulate hemiplegic lower limb weight-bearing asymmetry: (a) Resistance leash; (b) Wearing schematic.
4.1.1. Lower limb weight-bearing asymmetry simulation experiment
In this study, S01 (male, aged 22) was assigned to participate in the simulation experiment. The experiment simulated the weight-bearing asymmetry of the lower limbs of hemiplegic patients by setting a traction resistance cord on one side of the thigh shown in Figure 16. After wearing the resistance cord, the participant performed the sit-stand transfer movement, the specific experimental procedure was as follows: the participant sat naturally on the chair, and then started to stand up slowly; after completing the standing movement, wait for about 1s, and then the experimental operator started the EMG acquisition software to stop the acquisition and save the data, and the experiment repeats sit-to-stand movement for 10 times. Each side of the thigh was required to wear a separate resistance cord to counteract the effect of the dominant leg. Considering the muscle contribution rate [Reference Sadeghi, Allard and Duhaime91], six muscles on both lower limbs, named rectus femoris, lateral femoris and tibialis anterior, were selected as the sources for the experimental data acquisition as Figure 17, and the acquisition equipment and signal processing were the same steps as those in Sections 2.2 and 2.4.
Lower limb muscle group selection.
4.1.2. Analysis of the results of simulation experiments on lower limb asymmetry
The indicator for evaluating the weight-bearing asymmetry of the lower limbs during sit-to-stand transfer is WBasym (weight-bearing asymmetry) [Reference Lecours, Nadeau, Gravel and Teixera-Salmela57], which is equal to the average weight-bearing of the lower limbs on the affected side (nondominant side)/the average weight-bearing of the lower limbs on the healthy side (dominant side), and when the value is 1, it means that weight-bearing of the lower limbs of both limbs is completely symmetrical. The maximum amplitude value, average absolute value, integral EMG value and root mean square value of EMG signals reflect the energy size of EMG signals and physiological phenomena [Reference Nazmi, Rahman, Yamamoto, Ahmad, Zamzuri and Mazlan92]. In this paper, the feasibility of EMG will be analyzed by calculating the WBasym of these four indicators. Table XIV shows the results of symmetry analysis during autonomous standing, left-side weight-bearing standing and right-side weight-bearing standing.
As can be seen in Table XIV, the activity of the thigh muscles on both sides under weight-bearing standing showed significant asymmetry. Specifically, The WBasym values for the Rectus Femoris and Vastus Lateralis during weight-bearing standing showed symmetrical values with a clear asymmetric feature. In particular, this asymmetry was more pronounced when weight-bearing on the left side than on the right side, which may be related to the participants’ habitual use of the right leg. In contrast, asymmetry of the tibialis anterior muscle is reduced on both sides of the body during weightbearing, due to a significant increase in the activation state of the tibialis anterior muscle during tasks that require larger ankle stability (e.g., weightbearing, one-legged or unstable-surface standing), which reduces asymmetry on both sides [Reference Jeon, Jensen and Griffin93]. However, the asymmetry is still altered considerably compared to self-standing. These results indicate that the assessment method based on surface EMG signals is necessary and provides a basis for the assist effect assessment system established in this paper, and further supports the potential clinical evaluation by EMG signals in analyzing the functional asymmetry of the lower limbs and rehabilitation effects of stroke patients.
4.2. Limitations
By considering motion-related EEG and EMG signals and introducing a systematic approach and 1-DOF rehabilitation robot into rehabilitation training, this study analyzes the effectiveness of rehabilitation robot training based on BMI and promotes the reconstruction of neuromuscular pathways by providing users with neuromuscular feedback. Based on the results of the assist effect evaluation, the BMI module and the rehabilitation robot could be adjusted for continuous improvement of the patient’s rehabilitation strategy. EEG rhythms and EMG signals directly reflect human motion intentions and muscle states. The proposed method, based on the MRCP in MI in sit-to-stand transfer, confirms the effectiveness of MRCP in AO-MI through deep learning techniques for better classifying bidirectional motion intentions. Moreover, this study has evaluated the robot’s assist effect for sit-to-stand transfer. The results from the study have illustrated a better physiological experience of the proposed 1-DOF sit-to-stand transfer robot, which has a positive effect on the will of rehabilitation [Reference Eom and Lee94, Reference Wang, Hu, Yu, Liao, Zhang, Wu and Pei95]. The study has achieved better human-robot interaction and demonstrated the potential of this robot for sit-to-stand rehabilitation based on the results from assessments, which can further promote the rehabilitation training effect.
However, there are several limitations in the current work. Firstly, the study focused primarily on healthy participants and had a limited focus on lower limb muscles. Subsequent studies should focus on clinical scenarios and increase the number of participants, as well as onsite assessments of the effectiveness of the assist to validate and improve the utility of the system. Second, in discussing the necessity of EMG in the established BMI system, it emphasizes the preparatory phase of clinical trials and has not yet involved the impact of bilateral imbalance on the effectiveness of the assist. Future studies would conduct experiments on stroke patients to further validate the role of EMG in the BMI system and intensively analyze the influence of asymmetry on the effect of rehabilitation training, thus enhancing the technical and clinical application potential of the study. Thirdly, although the decoding model proposed in this paper shows good results in terms of decoding accuracy, it has a large storage demand and computational demand, which is mainly caused by the introduction of ResNet34, but the importance and effectiveness of the ResNet module is proved in the ablation experiments, and the combination of different modules and KANs can be attempted in subsequent studies to achieve a balance between the decoding accuracy and the storage and computational demands. Finally, although the single-degree-of-freedom sit-to-stand transfer robot established in this paper achieves a certain degree of effectiveness in the assist effect, if the robot’s degree of freedom increases, it may lead to a further decrease in coherence, as the reduced synergy between the central nervous system and muscles makes the rehabilitation more passive, which reduces the rehabilitation effect and prolongs the rehabilitation cycle. Future research should focus on evaluating the effectiveness of the single-degree-of-freedom sit-to-stand transfer robot in a more comprehensive manner, and compare the robot with other degrees of freedom in order to achieve a balance between the effectiveness of the rehabilitation and the intention of active rehabilitation.
Symmetry analysis of standing,S-F for self-standing,L-s for left side weighted standing, R-S for RF for right side weighted standing,Rectus Femoris,VL for vastus lateralis,TA for tibialis anterior.
5. Conclusion
In this study, a BMI system for a 1-DOF sit-to-stand rehabilitation training robot was established for human-robot interaction based on the actual rehabilitation training for the sit-to-stand transfer. Corresponding experiment paradigms were designed and the experimental results were analyzed, it proved that the MRCP in AO-MI was more suitable for sit-to-stand rehabilitation training. For human-machine interaction in sit-to-stand transfer, this study decodes bidirectional motion intentions from EEG signals. A Gumbel-Resnet-KANs model was constructed to deal with the non-linearity of EEG signals and to make full use of the MRCPs in AO-MI. The comparative and ablation experiments demonstrated that the model has significant performance advantages in the EEG signal classification task, especially in the bidirectional motion intentions of sit-to-stand transfer. The physiological effectiveness of the robotic rehabilitation was evaluated by considering the coherence of EEG-EMG signals, which illustrates the improvement of physiological feedback with the assistance of the robot. The psychological effectiveness of robotic rehabilitation was assessed by considering DASM and RASM, which illustrates the improvement of emotional levels with the assistance of the robot. This study provides a new EEG decoding approach for the BMI application in sit-to-stand assistive robots and an EEG-based assessment method for similar lower limb rehabilitation from a neuromuscular perspective.
Author contributions
CH and QC wrote the manuscript. CH implemented the machine learning methods. PC provided original ideas and suggestions for experiment design and assisted effect evaluation. XL designed the rehabilitation robot and KL analyzed the clinical problems. All authors contributed to the article and approved the submitted version.
Financial support
The study has been financially supported by the Sichuan Science and Technology Programme (Grant Nos. 2023YFH0037 and 2024YFFK0033), Chengdu Science and Technology Programme (Grant No. 2024-YF05-02675-SN), and 2023GJHZ0108),
$1\cdot 3 \cdot 5$
project for disciplines of excellence, West China Hospital, Sichuan University (Grant Nos. ZYYC21004 and ZYJC21081), and
$1\cdot 3\cdot 5$
projects for Artificial Intelligence (Grant No. ZYAI24005), West China Hospital, Sichuan University.
Competing interests
All contributors agreed and given consent to publication.
Ethical approval
The study involving human participants was reviewed and approved by the Ethics Committee of West China Hospital of Sichuan University (2022-670).
