1. Introduction
Immersive virtual reality (VR) aims to create a highly engaging and interactive environment that closely replicates real-world experiences. Recent advancements in hand-tracking technologies have improved the overall user experience regarding hand–object interaction; however, several limitations remain in achieving realistic manipulation. In addition to inaccurate hand modeling, simplified contact dynamics, and unstable object interactions (Mangalam et al., Reference Mangalam, Oruganti, Buckingham and Borst2024), the lack of realistic tactile feedback remains an important bottleneck. The missing feedback directly constrains the user’s ability to perceive and regulate interactions with virtual objects limiting the sense of presence and embodiment in VR (Kilteni et al., Reference Kilteni, Groten and Slater2012). Despite recent adoption in commercial systems, current haptic technologies provide only limited approximation of the natural hand–object interaction, leaving a substantial gap between virtual and real-world experience. In natural manipulation, the numerous mechanoreceptors distributed across the glabrous skin of the human hand provide crucial information that allows for precise and dexterous control, such as surface properties or contact force distribution (Johnson et al., Reference Johnson, Yoshioka and Vega-Bermudez2000; Johansson and Flanagan, Reference Johansson and Flanagan2009). Natural tactile feedback is spatially distributed and high resolution, and as such, difficult to replicate using wearable haptic feedback systems. Mechanotactile interfaces provide modality-matched feedback by applying direct pressure to the skin through actuated indentations, either perpendicularly or by inducing lateral stretch (Basdogan and Srinivasan, Reference Basdogan and Srinivasan2002; Choi et al., Reference Choi, Culbertson, Miller, Olwal and Follmer2017; Leonardis et al., Reference Leonardis, Solazzi, Bortone and Frisoli2017). Despite their effectiveness, these systems rely on complex mechanical components, making them bulky and less portable and limiting their integration into lightweight VR setups. Vibrotactile stimulation systems, in contrast, are particularly popular haptic interfaces due to their simplicity and relatively low cost. They deliver mechanical vibrations to the skin, typically within a frequency range that activates fast-adapting mechanoreceptors (Islam and Lim, Reference Islam and Lim2022). Recent design innovations have led to the development of miniature vibrotactile devices that seamlessly adhere to the dorsum of the fingertip and incorporate embedded electronics (Preechayasomboon and Rombokas, Reference Preechayasomboon and Rombokas2021), further decreasing the form factor. However, vibrotactile systems struggle in leveraging the high spatial and temporal resolution provided by the dense network of mechanoreceptors in the human hand. Most commercially available hand-based haptic devices relying on vibrotactile stimulation can provide at most a single actuation point per finger (e.g., SenseGlove Nova, Manus Metagloves Pro Haptic), which inherently limits spatial resolution and, consequently, the realism of the perceived tactile feedback.
Electrotactile interfaces offer a potential solution to these limitations. These systems directly stimulate the nerve fibers in the skin, bypassing natural transducers and still evoking tactile sensations in targeted areas. The electrical current travels through the subdermal space between the anode and cathode, stimulating the skin afferents below the electrode (Ghafoor et al., Reference Ghafoor, Kim and Hong2017). Because of the artificial and non-selective nature of the stimulus (i.e., the direct stimulation of the nerves rather than the transduction from mechanical to electrical energy), the quality of the evoked sensation is different than those produced by natural stimuli. Nevertheless, the ability to finely and independently tune the parameters allows for the rendering of diverse tactile sensations (Kourtesis et al., Reference Kourtesis, Argelaguet, Vizcay, Marchal and Pacchierotti2022). Furthermore, customized solutions can be developed for targeted applications by optimizing the system’s design. Electrodes can be manufactured with varying numbers and arrangements of stimulation pads, which are printed on flexible materials (Štrbac et al., Reference Štrbac, Belić, Isaković, Kojić, Bijelić, Popović, Radotić, Došen, Marković, Farina and Keller2016; Malešević et al., Reference Malešević, Isaković, Garenfeld, Došen and Štrbac2021; Isakovic et al., Reference Isakovic, Malesevic, Kostic, Dosen and Strbac2022) and with a small inter-pad distance that allows for high spatial resolution, limited only by two-point discrimination threshold (e.g., as low as ~2 mm at the fingertips) (Solomonow et al., Reference Solomonow, Lyman and Freedy1977, Reference Solomonow, Raplee and Lyman1978). In previous investigations, compact multi-pad electrodes were used to stimulate the fingertip, and the participants could discriminate the position of individual pads (Bobich et al., Reference Bobich, Warren, Sweeney, Helms Tillery and Santello2007; Warren et al., Reference Warren, Bobich, Santello, Sweeney and Tillery2008; Isakovic et al., Reference Isakovic, Malesevic, Kostic, Dosen and Strbac2022; Garenfeld et al., Reference Garenfeld, Henrich, Isaković, Malešević, Štrbac and Došen2023) and recognize spatial patterns (Kaczmarek and Haase, Reference Kaczmarek and Haase2003; Kajimoto et al., Reference Kajimoto, Kawakami, Tachi and Inami2004), despite the very limited skin area (Henrich et al., Reference Henrich, Garenfeld, Malesevic, Strbac and Dosen2024). These findings show that certain features of natural feedback, such as high fidelity and spatial distribution, can be approximated by modulating electrotactile stimulation parameters.
Electrotactile feedback has been employed in VR applications to recreate tactile sensations, both to enhance the embodiment of the virtual hand (Fröhner et al., Reference Fröhner, Beckerle, Endo and Hirche2019; Garenfeld et al., Reference Garenfeld, Jiménez-Díaz, Navarro-Moreno, Tormo, Štrbac, Hernández, Baños, Herrero and Došen2024) and to convey grip force or object physical properties in active manipulation (Hummel et al., Reference Hummel, Dodiya, Center, Eckardt, Wolff, Gerndt and Kuhlen2016; Pamungkas and Ward, Reference Pamungkas and Ward2016; Yem et al., Reference Yem, Vu, Kon and Kajimoto2018). However, only a few studies have explored multipoint stimulation at the fingertip, despite its potential to reproduce richer tactile cues. Existing work has focused on recreating complex sensations, such as softness–hardness perception (Yem et al., Reference Yem, Vu, Kon and Kajimoto2018), or simpler cues, such as contact force (Hummel et al., Reference Hummel, Dodiya, Center, Eckardt, Wolff, Gerndt and Kuhlen2016; Vizcay et al., Reference Vizcay, Kourtesis, Argelaguet, Pacchierotti and Marchal2023). In these studies, tactile information was encoded either by modulating the stimulation intensity (Kajimoto et al., Reference Kajimoto, Kawakami, Maeda and Tachi2001; Pamungkas and Ward, Reference Pamungkas and Ward2016), pulse frequency (Hummel et al., Reference Hummel, Dodiya, Center, Eckardt, Wolff, Gerndt and Kuhlen2016), or pulse width (Vizcay et al., Reference Vizcay, Kourtesis, Argelaguet, Pacchierotti and Marchal2023). Among these strategies, intensity-based encoding is often considered the most intuitive, as the stimulation current can be directly mapped to the perceived intensity of the sensation. However, this approach is also particularly susceptible to discomfort: if not properly calibrated, increased stimulation amplitudes may activate nociceptive fibers, leading to unpleasant or even painful sensations (Delitto et al., Reference Delitto, Strube, Shulman and Minor1992). One promising alternative approach is to encode grasp force through spatial mapping, which leverages the high resolution of electrotactile interfaces and may also be perceived as more natural. In fact, increasing the number of active pads mimics the spatiotemporal behavior of parallel afferent fibers – where a larger contact area stimulates an increasing number of sensory fibers – leading to a more intense and widespread sensation (Johansson and Flanagan, Reference Johansson and Flanagan2009). Despite its potential, spatial encoding of grasp force at the fingertip has not yet been systematically investigated, nor have different encoding strategies been directly compared.
Therefore, how to best encode the information about the grasping force when using high-density electrotactile stimulation to maximize performance and user experience is still an open question. In the present study, we explored different strategies for mapping the grasp force using electrotactile feedback within an immersive VR environment and compared their effectiveness in improving object manipulation. A fully wearable system featuring multipoint electrodes was employed to deliver stimuli to the thumb and index fingertips while participants performed a force control, pick-and-place task. Electrotactile feedback was modulated across spatial and temporal domains to assess which strategy most effectively conveyed the virtual grasp force. Our goal was to determine the optimal encoding method to translate the grasping force into stimulation profiles and establish the benefits of artificial feedback compared to the absence of tactile cues. Ultimately, this research aims to make electrotactile solutions more suitable for real-world applications by improving users’ ability to perceive and interact with virtual objects, thereby potentially enhancing performance, immersion, realism, and the sense of embodiment.
2. Material and methods
2.1. Experimental setup
The Oculus Quest 2 (Meta Platforms, USA) headset was used to show the immersive virtual scene to participants and track their hand movements through the built-in camera-based hand-tracker. The VR scene was developed using Unity (Unity Technologies, USA) and was synchronized with the electrotactile system through a Bluetooth connection (65 Hz rate). The electrotactile stimulation was delivered through the same system used in (Garenfeld et al., Reference Garenfeld, Jiménez-Díaz, Navarro-Moreno, Tormo, Štrbac, Hernández, Baños, Herrero and Došen2024), which comprised compact 6.5 × 4.5 × 3.5 cm 32-channel stimulator (Tecnalia Research and Innovation, Spain) and flexible strips embedding multi-pad electrodes at the distal end (Tecnalia, Serbia). Two identical electrodes were positioned on the fingertips of the thumb and index fingers of the dominant hand. Each electrode integrated a 3 × 2 matrix arrangement of active pads (single pad: 2 × 3.5 mm, intra-pad distance: 1 mm) enclosed by an H-shaped return electrode (Figure 1). The stimulator generated symmetric biphasic rectangular pulses asynchronously separated by a fixed 500 μs inter-pulse interval. Cathodic-first stimulation waveforms were generated as they induce a more selective sensation in the active electrode (Szeto and Saunders, Reference Szeto and Saunders1982; Kajimoto, Reference Kajimoto, Kajimoto, Saga and Konyo2016). Pulse frequency was a global parameter common to all pads, while pulse amplitude and width could be modulated individually for each pad in the range of 1–200 Hz, 0.1–9 mA, and 50–500 μs with 1 Hz, 0.1 mA, and 10 μs increments, respectively. In the present experiment, the pulse width was fixed at 100 μs.
The experimental setup included the headset and the electrotactile system, comprising the multichannel stimulator and the flexible electrodes for thumb and index fingertips. Each electrode integrated a 3 × 2 matrix arrangement of active pads enclosed by an H-shaped reference pad.
Figure 1. Long description
Each electrode integrated a 3 × 2 matrix arrangement of active pads enclosed by an H-shaped reference pad, with the active pads of the electrodes are positioned on the most distal phalanx of the finger.
2.2. Grasping force estimation
The VR task involved grasping and moving a virtual cube while applying the target level of force (dynamic force matching task), as explained in Section 2.4.5 (Figure 2a). The virtual force applied by the participant to the object was estimated by measuring the distance between the bones of the thumb and index fingertips of the virtual hand when they both made contact with the two opposite faces of the cube (Figure 2b). The distance was normalized over the length of the cube’s side, and the grasp force GF was computed as:
where D is the distance between the bones and L is the cube side length. Essentially, the force was proportional to the amount of interpenetration of the fingers into the object and always between 0 and 1. While this represents an oversimplification of natural physical interactions – since in reality the pressure-force relationship of the human hand is nonlinear – it is sufficient for the purposes of this study, which focuses on comparing different encoding schemes for electrotactile feedback. The size of the virtual cube (L ≈ 8 cm) provided a consistent reference for interpreting this distance: for example, a 1 cm reduction in fingertip distance (D) during grasping would result in a 12.5% increase in the computed force. The cube dimension was a compromise between maximizing the distance between fingertips, and thus the range available to modulate the force, and ensuring a comfortable grasp across a large variation of participants with varying hand sizes. Six force levels were defined, each corresponding to a specific range of normalized fingertip distance values. The thresholds that separate the force levels were spaced to be approximately equally distributed between 0.1 and 0.9 (i.e., 0.1, 0.23, 0.36, 0.49, 0.62, and 0.75). The lower margin (0.1) prevented the system from immediately registering force upon initial contact, which might seem counterintuitive, as tactile perception typically begins at the moment of touch. However, this approach allowed each level to have both a lower and an upper boundary, which was necessary to compute the success rate of force control (see Section 2.5). Similarly, the upper limit (0.9) prevented the user from simply pressing their fingertips together to reach the highest force level, which would make force control for this level trivial as the participants could rely on the natural tactile feedback (touch between the fingertips). To prevent errors caused by the hand-tracking performance, the thresholds defining each force level were adjusted online. Pilot testing showed that the accuracy in estimating the fingertips position was affected by the relative positioning of the camera and the hand, causing the system to erroneously detect changes in force levels that did not correspond to actual finger movements. To mitigate this, once the user reached the target force level, the corresponding threshold range was widened by a small margin of 0.08 (i.e., ±0.04 around the target value). Specifically, all thresholds above the current level were shifted upward by 0.04, while all lower thresholds were shifted downward by 0.04 (Figure 2c). The value of the margin was determined through pilot testing. This adjustment stabilized control, making it easier for the user to maintain the correct force level despite potential tracking noise. The thresholds were reset to their original values as soon as the current force exited the target level.
(a) The virtual reality task involves reaching, grasping, and moving a virtual cube while applying a target level of force. The target force is indicated both by a background legend and by the color of the static reference cube marking the target position. (b) The grasping force is computed as the distance between the thumb and index fingertips with respect to the cube side length. (c) The grasping force is discretized into six levels, separated by defined thresholds (0.10, 0.23, 0.36, 0.49, 0.62, and 0.75). These thresholds are adjusted online to balance hand-tracking performance and control stability by widening the force level range when the applied force matches the target level. The force estimated from the fingertip distance (raw grip force) is shown as the orange line, whereas the blue line indicates the discrete level. Thin grey lines are the thresholds defining the levels. Note that when the estimated force crosses the threshold for level 2, the level width (shaded region) becomes temporarily wider. Task progress (top panel in c) is the normalized distance to the target point.
2.3. Electrotactile feedback encoding
Participants were provided with online feedback about the current force level (ranging from levels 1 to 6). The electrotactile feedback strategies were developed based on principles of human tactile perception, aiming to provide an effective and intuitive representation of grip force. Three strategies were implemented: frequency modulation, spatial modulation, and a mixed modulation approach that combined both. The stimulation intensity was constant and set to a localization threshold determined for each pad and participant individually, as explained in Section 2.4.3. A graphical representation of the encodings is shown in Figure 3.
Active pads and stimulation frequency associated with each force level across the different encoding schemes. Higher forces are conveyed by (1) increasing the frequency of the two middle pads (left), (2) activating more pads (middle), and (3) by combining the two previous approaches.
2.3.1. Frequency modulation
This strategy leveraged the nervous system’s ability to interpret temporal variations in stimulation, and it also reflects the increase in the frequency of firing of slowly adapting receptors in response to higher forces (rate coding). Each force level was conveyed by activating the two middle pads at a specific stimulation frequency, with frequencies set at 8, 15, 26, 40, 70, and 100 Hz, respectively. The frequency values were selected based on the discriminative perception curve reported in (Štrbac et al., Reference Štrbac, Belić, Isaković, Kojić, Bijelić, Popović, Radotić, Došen, Marković, Farina and Keller2016). Minor adjustments were made to the lower frequencies to ensure the pulse rate was not too low, preventing the timely delivery of the feedback.
2.3.2. Spatial modulation
This approach aimed to mimic the natural increase in contact area that occurs during physical object manipulation. Each force level was indicated by a correspondent number of active pads, with lower forces activating fewer pads in a predefined order. Pulse frequency was fixed at 50 Hz.
2.3.3. Mixed modulation
This strategy combined frequency and spatial modulation, with each force level associated to a specific pulse frequency (same as for frequency modulation) and a corresponding number of active pads (same as for spatial modulation). Therefore, as grip force increased, the stimulation became simultaneously faster and more spatially distributed, hence mimicking the two physiological mechanisms of force encoding (namely, spatial recruitment and rate coding).
2.4. Experimental protocol
2.4.1. Participants
Twelve adults with unimpaired hand and arm function and naïve to the purpose of the experiment (age: 28.83 ± 4.17 yrs., nine males and three females, all right-handed) participated in the study. The experiment was conducted in accordance with the Declaration of Helsinki, and informed consent was obtained from each participant before commencing the experiments. The study was approved by the Research Ethics Committee of the Nordjylland Region (approval number N-20190036).
2.4.2. Experimental procedure summary
Participants sat comfortably in a chair in front of a desk, wearing the head-mounted display. After calibrating stimulation intensity, they performed the VR task without receiving any tactile feedback (pre-no-feedback), establishing a baseline to evaluate their performance without electrotactile assistance (Figure 4). Subsequently, they repeated the experimental protocol, including both the electrotactile feedback discrimination and the VR task (both explained below), for each of the three tactile feedback modalities (frequency, spatial, and mixed modulation). The order of these modalities was pseudo-randomized to counterbalance potential learning effects across participants. Finally, to assess any motor learning effects, the validation block of the VR task was repeated without tactile feedback at the end of the experiment (post-no-feedback). After performing each condition, subjective measures were acquired from a 12-item questionnaire (see Supplementary Materials). Items Q1–Q8 assessed the sense of presence and embodiment (ownership, location, and agency) in the VR task and were adapted from Schubert et al. (Reference Schubert, Friedmann and Regenbrecht2001) and Crone and Kallen (Reference Crone and Kallen2024), while items Q9–Q12 assessed tactile sensations and were created ad hoc to ensure a comprehensive assessment. The participants reported on a 7-point Likert scale how much they agreed with each statement (1 is “strongly disagree” and 7 is “strongly agree”). At the very end of the experiment, they were asked to rank the conditions according to their personal preference in terms of comfort and perceived performance (1 was “most preferred” and 4 was “least preferred”). The experimental phases including the calibration, electrotactile feedback discrimination, and VR force-matching task are explained below.
Pipeline of the experimental protocol. After calibrating the intensity of electrotactile stimulation, the participants performed the task without tactile feedback (pre-block), with tactile feedback using three encoding methods in pseudo-random order, and one more time with the feedback deactivated (post block). For each feedback method, the participants performed a tactile feedback discrimination test before proceeding to the task. After each block except the last one (post feedback), they were asked to answer the questionnaire.
2.4.3. Calibration
The electrodes were carefully strapped on the thumb and index fingertips using medical tape. Special care was taken to avoid any restriction of blood circulation, as this could lead to sensations such as pulsation or numbness, and potentially affect the perception of the stimulation. The pulse amplitude of each active pad was determined at the beginning of the experiment using the ascending method of limits, starting from 500 μA and then increasing the amplitude in steps of 100 μA. The amplitude increment was manually controlled by the experimenter by clicking a virtual button. During this process, participants provided verbal feedback when they detected a clear, localized sensation beneath the activated pad (Malešević et al., Reference Malešević, Isaković, Garenfeld, Došen and Štrbac2021). After each pad was calibrated, all 12 pads were activated sequentially, and the stimulation intensity was individually fine-tuned to elicit clear and localized sensations that were similar across pads. Pulse width and frequency were kept constant throughout the calibration session at 100 μs and 50 Hz, respectively. The calibration procedure lasted approximately 10 mins.
2.4.4. Electrotactile feedback discrimination
Participants received electrotactile stimulation corresponding to specific force levels, mapped through the tested modality (frequency, spatial, or mixed), and were asked to identify the force level they perceived. The test consisted of 12 trials, with each of the six force levels presented twice as the target in random order. To simulate real task conditions, the test always began at force level 1, and then the experimenter gradually increased the force level through all intermediate levels until the target was reached (simulating the gradually increasing pressure applied by the fingertips to the grasped object). Once the target level was reached, participants were asked to verbally indicate which force level was provided. This test served both to familiarize participants with the feedback and to allow the experimenter to refine the calibration in cases of poor performance. For one participant, the test was interrupted after a few trials due to inadequate discrimination performance; the system was subsequently recalibrated, and the test was repeated. Data from the initial attempt were excluded from the analysis.
2.4.5. VR force-matching task
The VR experiment consisted of three blocks: familiarization, reinforced learning, and validation. The familiarization block consisted of 15 trials, allowing participants to get familiar with both the electrotactile feedback and the motor task. In each trial, participants were asked to grasp the virtual cube using the target level of force. Participants were required to use a pinch grasp, involving only the thumb and index finger. The target force level was visually indicated using a color map, with a corresponding color legend displayed in the background of the virtual scene throughout the experiment (Figure 2a). Only force levels from 1 to 5 were provided as targets, intentionally excluding level 6 (the maximum possible force) to avoid giving participants a clear upper bound and prevent them from simply squeezing the cube as hard as possible to reach the highest level. In this phase, the cube was static, and the goal for the participants was to understand how the force increased when pressing on the cube and how it was mapped to the tactile feedback. During the first five trials, participants received both visual (i.e., the cube changed color accordingly with the applied force) and tactile feedback. For the remaining 10 trials, only tactile feedback was provided, requiring participants to rely solely on electrotactile sensations. The reinforced learning block consisted of 20 trials, where participants were tasked to move the virtual cube toward a designated target point (distance ≈40 cm) while applying the target force level. At the beginning of each trial, a static and non-interactive cube appeared first, marking the target position and indicating the target force level through its color. Then, the cube that was supposed to be moved by the participant was visualized at the initial position. They were instructed to reach and grasp the cube and produce the correct target force before moving the cube from its initial position. In fact, they were allowed to adjust the applied force within a small “safe zone” near the starting point – 10% of the distance toward the target position – without failing the trial. Unlike the familiarization block, feedback related to the current force level was provided only through tactile sensations, as visual cues were removed. To successfully complete a trial, participants had to maintain the target force level while transporting the cube from the initial to the target position. If the applied force deviated from the target for more than two consecutive seconds, the trial was considered a failure. In particular, if the applied force was lower than the target force level, the cube would drop, while excessive force would cause the cube to explode, and the scene would automatically progress to the next trial. Finally, the validation block was structured similarly, but consisted of 30 trials. In this block, participants were not informed about any task failures. Even if the incorrect force was applied, they could still complete the trial and move the cube to the target point without being informed about whether the force was too low or too high (no drop or cube explosion).
2.5. Metrics and data analysis
Both objective and subjective metrics were extracted. From the electrotactile feedback discrimination, the number of correct answers for each condition was recorded. From the VR force-matching task, time-related metrics, accuracy, deviation from the target force level and success rate upon exiting the safe zone were computed. For each of the 30 validation trials, the time spent in the safe zone (i.e., the time spent adjusting the force applied to match the target force) and the time taken to effectively move the cube to the target were extracted. Both the accuracy and the deviation from the target were computed only during the moving phase. Specifically, the accuracy was defined as the total time during which the current force level matched the target force level expressed in percent of transfer time, while the deviation was calculated as the average absolute difference between the target and the current force levels when the force level was not matching the target force. The accuracy indicates how successful participants were in maintaining the correct force, whereas deviation measures how far they deviated from the target force (when they were not successful). Instead, the success rate upon exiting the safe zone was defined as the percentage of trials in which participants were applying the correct force level at the moment they began moving the cube. The average values of all outcome measures across trials were computed for each participant and condition.
Statistical analyses were conducted on both objective and subjective metrics to evaluate significant differences between the feedback conditions. Kolmogorov–Smirnov test revealed that the data were not normally distributed for any of the metrics. Consequently, the Friedman test was conducted first to assess the group-level difference, and if significance was detected, the post hoc pairwise tests were performed using the non-parametric Wilcoxon signed-rank test. To account for multiple comparisons and reduce the risk of Type I errors, the Bonferroni correction was applied. Data analysis was conducted using MATLAB 2024b (MathWorks, USA). The results in the text are reported as median and interquartile ranges (IQR) in brackets.
3. Results
The trajectories shown in Figure 5 illustrate representative examples of one trial when receiving tactile feedback using the mixed modulation (on the left) and when the tactile feedback was deactivated (on the right, pre-no-feedback test). In these trials, the condition with feedback yielded an accuracy of 58.8% and a deviation of 1. In contrast, the no-feedback condition resulted in an accuracy of 0% and a deviation of 2.18. When the feedback was available, the participant gradually increased the grasping force to the correct level, and then exited the safe zone and started moving the cube toward the target position. At one point, the force became too high, but the participant noticed and corrected this deviation before the end of the trial. This demonstrates a successful reaction due to the feedback both in the safe zone and during the movement. Without feedback, there was no gradual “staircase” increase in the safe zone, and the generated force was off with respect to the target consistently throughout the whole trial. However, feedback-driven control also increased the time to perform the task. Specifically, the time spent in the safe zone and the full trial duration were 1.42 and 4.14 s with feedback, and 0.66 and 3.17 s without it, respectively.
Representative examples of a single trial are shown for two conditions: with tactile feedback using mixed modulation (left) and without tactile feedback (right, pre-no-feedback test). “Raw force” refers to the distance between the thumb and index fingertips, from which the current level of force is estimated. “Task progress” represents the movement trajectory toward the target. When the feedback is available (left), the participant gradually increases the force in the safe zone and then maintains it within the correct level during movement by noticing and correcting deviations when necessary.
3.1. Electrotactile feedback discrimination
Electrotactile feedback discrimination was performed prior to the VR force-matching task (i.e., before any training). The post hoc pairwise comparison of the obtained success rates (Figure 6a) following a Friedman test showed it was significantly more difficult to discriminate the feedback levels in the frequency encoding (54.2 [41.7, 62.5]%) compared to the spatial (83.3 [66.7, 95.8]%, p < 0.05) and mixed encoding (91.7 [75.0 100]%, p < 0.01), while there was no significant difference between the latter two methods. The results show that the frequency encoding was intrinsically more difficult to recognize. Nevertheless, after the discrimination test, the participants received further training (familiarization and reinforced learning) before proceeding with the actual validation.
(a) Success rate of the electrotactile feedback discrimination test; (b) accuracy in matching the target force level; (c) success rate upon exiting the safe zone; and (d) deviation from the target force level. Spatial and mixed encoding were easier to discriminate compared to frequency modulation without any training. The feedback increased the performance of the VR force control task, especially when using spatial and mixed encoding methods.
3.2. VR task-related metrics
The accuracy values (Figure 6b) in all conditions were well below 100%, which confirms that the task was not trivial. The median accuracy was consistently higher with feedback compared to the no-feedback conditions. However, only the spatial and mixed encoding schemes showed statistically significant improvements compared to both pre- and post-no-feedback blocks. No statistically significant differences were observed between the feedback conditions. However, trend-wise, participants achieved higher median accuracy with the mixed encoding, reaching an accuracy of 70.6 [56.5, 80.9]%, compared to the spatial (59.2 [50.3, 71.6]%) and frequency (56.3 [41.5, 64.4]%) modulation. The median accuracy in no-feedback conditions was similarly low pre (39.7 [32.3, 56.1]%) and post (44.4 [33.5, 60.5]%). For the safe zone success rate, mixed encoding resulted in the highest median success rate (76.7% [63.3, 90.0]), followed by spatial (61.7% [51.7, 78.3]) and frequency (56.7% [46.7, 85.0]) encoding (Figure 6c). All feedback modalities showed statistically significant differences compared to the no-feedback conditions, with the exception of the frequency scheme relative to the pre-no-feedback block. Additionally, the mixed modality significantly outperformed the spatial encoding scheme. Deviations from the target force level followed a trend similar to that observed for the accuracy metric, as the median deviation from the target force was consistently lower when electrotactile feedback was provided (Figure 6d). Statistical analysis revealed significant differences between mixed modality and no-feedback modalities (both pre- and post-), but not across encoding schemes. Nevertheless, all feedback conditions showed trends toward lower deviations compared to no feedback. Regarding the time spent completing the task (Figure 7a), participants spent more time in the safe zone when provided with feedback (as also observed in the representative trials in Figure 5). All feedback conditions were statistically different from the no-feedback conditions (both pre- and post-), while no significant differences were observed across the feedback conditions. Interestingly, this pattern did not apply to the time spent actively moving the cube: in this case, the time was more similar across conditions, and the feedback significantly increased the movement time only compared to the post-no-feedback condition block (Figure 7b).
(a) Time spent in the safe zone during the task. (b) Time spent moving toward the target (moving time). The feedback increased the time to perform the task, but only during safe zone adjustments.
3.3. Subjective metrics
No statistically significant differences were found between conditions regarding the feeling of presence (Figure 8a). Mixed encoding increased (p < .024) the feeling of embodiment compared to no feedback (Figure 8b). No statistically significant differences were found between the feedback conditions related to the tactile sensation scores, and the sensation quality was rated generally high, namely 5.0 [4.9, 6] for frequency and 6.4 [5.6, 6.6] and 6.4 [5.8, 6.8] for spatial and mixed encoding, respectively (Figure 8c). Finally, when asked to provide a subjective ranking of the different modalities, participants showed a similar preference for the spatial and mixed encoding schemes, while the frequency encoding was almost always ranked third, but still before the no-feedback condition (Figure 8d).
Subjective metrics. Answers from the questionnaire were divided into four categories: (a) presence, (b) embodiment, (c) tactile sensation, and (d) ranking of the conditions (1 – best, 4 – worst). Mixed encoding increased the feeling of embodiment. All feedback conditions were preferred by the participants compared to no feedback, and spatial and mixed encoding were ranked better than frequency modulation.
Figure 8. Long description
Mixed encoding increased the feeling of embodiment. All feedback conditions were preferred by the participants compared to no feedback, and spatial and mixed encoding were ranked better than frequency modulation. Participants preferred the mixed encoding, followed by spatial, frequency and no feedback condition.
4. Discussion
In this study, we investigated strategies for conveying grasp force through multipoint electrotactile feedback in an immersive VR environment, focusing on their effectiveness in enhancing a pick-and-place task. Feedback was delivered to the thumb and index fingertips via three encoding schemes, namely, frequency modulation, spatial modulation, and a mixed (spatial + frequency) modality, and we evaluated both objective performance and subjective preferences. The goal was to 1) investigate the utility of tactile feedback for manipulation tasks in VR, and 2) determine which encoding strategy offered superior perceptual effectiveness by leveraging the temporal and/or spatial resolution of human tactile perception.
The results showed that providing tactile feedback seemed to be generally beneficial, thereby reinforcing the importance of tactile cues in immersive VR environments (Apostolou and Liarokapis, Reference Apostolou and Liarokapis2022). However, the experiments also demonstrated that the encoding approach is an important factor, as the performance improved significantly compared to no feedback only when using mixed and spatial encoding. Importantly, the results for the safe zone success rate indicate that the feedback was beneficial for initial adjustment of the force, while the object was static, which presumably allowed the participants to fully focus on the feedback cues. The improved accuracy suggests that feedback remained effective during the movement phase, despite the added challenge of controlling force while moving. Finally, as shown by the deviation, when the participants went off the target force, they made smaller deviations when the feedback was available.
The fact that tactile feedback, when properly encoded, was useful for force control is particularly relevant given the possibility that proprioceptive feedback alone might have been sufficient for estimating grasp force. In fact, the grasp force was determined by the distance between the fingertips, and therefore, the participants could use natural proprioception to adjust force, potentially rendering additional tactile input redundant or even counterproductive by increasing task complexity. However, our results suggest that electrotactile feedback not only complemented proprioceptive cues but also enhanced users’ ability to finely modulate their grip. This aligns with prior research, that showed that enriching VR environments with multisensory stimuli, beyond the traditional visual and auditory channels, can significantly boost users’ presence, engagement, and task performance (Apostolou and Liarokapis, Reference Apostolou and Liarokapis2022).
Among the tested modalities, the mixed feedback scheme seemed to be the most effective approach since it significantly improved the accuracy and deviation compared to no feedback, and also outperformed spatial encoding in terms of safe zone success rate. This suggests that the combination of spatial and temporal cues enhanced the discrimination of force levels, further supporting the idea that modulating multiple parameters leads to improved stimulus discrimination (D’Alonzo et al., Reference D’Alonzo, Dosen, Cipriani and Farina2014; Seminara et al., Reference Seminara, Fares, Franceschi, Valle, Strbac, Farina and Dosen2020). This was further corroborated by the electrotactile feedback discrimination, where participants, even without prior training, achieved >90% correct responses with the mixed modality, significantly outperforming the frequency condition (54%, p < 0.01). Notably, the spatial modality also yielded strong performance (83%), highlighting the potential of spatial encoding in electrotactile feedback design. These positive results may, in part, reflect the alignment of spatial modulation with natural tactile perception, where increasing force leads both to a larger contact surface and to higher pressure magnitude. This principle has previously been employed to encode material softness, using mechanical haptic interfaces that simulate the increasing of contact area with the fingers to convey compliance information (Bicchi et al., Reference Bicchi, Scilingo and De Rossi2000). Building on this concept, our group has previously investigated the suitability of spatial encoding in closed-loop prosthetic control. In Dosen et al. (Reference Dosen, Markovic, Strbac, Belic, Kojic, Bijelic, Keller and Farina2017), both spatial and mixed encoding schemes were found to be effective in conveying grasp force information, further supporting the viability of spatial modulation for feedback applications. Additionally, in Garenfeld et al. (Reference Garenfeld, Mortensen, Strbac, Dideriksen and Dosen2020), spatial encoding was compared with amplitude modulation for delivering proprioceptive feedback related to wrist rotation and hand aperture, yielding comparable performance across both approaches.
However, it is interesting to note that the high performance achieved in the feedback discrimination for the spatial modality was not replicated in the VR task. Specifically, the success rate at the moment participants exited the “safe zone” – where they freely adjusted their force to match the target – was significantly lower (p < .01) for the spatial modulation condition (61.7%) compared to the mixed approach (76.7%). This suggests that while spatial modulation levels were easily identifiable when the task focused solely on discrimination, performance declined when participants had to combine this feedback into a more complex motor task (p < 0.05). In contrast, this drop in performance between discrimination and safe zone success rate was not observed for the mixed condition (p > 0.2), indicating that the additional temporal information (combined with spatial cues) provided a more robust feedback, which supported participants in maintaining a higher level of performance even when the task difficulty increased (from static perception to active closed-loop control).
Although intensity modulation is commonly tested encoding scheme in electrotactile feedback (Fleury et al., Reference Fleury, Lioi, Barillot and Lécuyer2020), it was deliberately excluded in this study. Modulating stimulation amplitude or pulse width would have increased calibration time as it needs to be performed individually for each pad, and it would also introduce a higher risk of discomfort (Garenfeld et al., Reference Garenfeld, Mortensen, Strbac, Dideriksen and Dosen2020). Both these factors are important limitations for the potential translation of this approach to real-world applications, and from that perspective, intensity modulation is a less attractive approach. Nevertheless, the lack of direct comparison with amplitude modulation is a limitation of the present study. Future work should therefore consider a systematic evaluation between amplitude modulation and the best-performing mixed encoding scheme, as this would be an important step in providing a complete benchmarking of electrotactile encoding strategies. Importantly, despite the long duration of the experiment (approximately 3 hrs in total), no participant reported discomfort and no one withdrew, demonstrating the comfort and feasibility of the chosen encoding strategies. Indeed, it should be noted that spatial modulation can inherently increase the overall intensity of the elicited tactile sensation due to the potential summation of sensations generated in response to individual pulses injected through multiple pads. A similar concern holds for frequency modulation, where higher stimulation rates may result in stronger sensations (Paredes et al., Reference Paredes, Dosen, Rattay, Graimann and Farina2015). Consequently, participants may have used overall intensity as an additional implicit cue. To isolate the contribution of pure spatial encoding, one would need to scale down the individual pad amplitudes when multiple pads are simultaneously active (or, similarly, when frequency is increased), but this remains to be investigated.
As expected, all the tactile feedback conditions were associated with longer task completion times. This delay likely reflects the additional cognitive processing required to interpret and integrate the tactile cues into the motor control scheme (Körding and Wolpert, Reference Körding and Wolpert2006). However, no significant differences in task duration were found across the three feedback modalities, indicating that the improved performance with the mixed scheme did not require increased effort or processing time. Interestingly, the increase in task duration was limited to the time spent in the “safe zone,” while the movement time remained unaffected (no significant difference between feedback and pre-no-feedback condition). This suggests that the challenging aspect of the task was more about matching the correct level of force than maintaining it while moving the cube. This could be related to the fact that during the initial adjustment, participants needed to “find” the correct level by increasing the force from zero to the target, whereas during the movement, they only needed to detect and correct the deviation from the target force using feedback. A small but significant decrease of the movement time in the post-no-feedback condition (always executed at the end of the experiment) was most likely attributable to fatigue and tiredness, as the pre-task movement duration was comparable to those under feedback conditions (p > 0.5). A related perspective is provided by Furmanek et al. (Reference Furmanek, Mangalam, Lockwood, Smith, Yarossi and Tunik2021), who provided contact information in a high-fidelity VR environment using visual and auditory sensory substitution. Their results showed that the type of feedback modality had little effect on task execution timing, demonstrating the potential of haptics-free VR. It would be relevant to compare these two approaches directly to establish the pros and cons of tactile versus haptics-free sensory substitution (vision and sound cues). Tactile feedback might penalize time, at least initially, but promote embodiment and realism. Nevertheless, this remains to be tested in future work.
Regarding subjective metrics, participants generally accepted the feedback and expressed positive attitudes toward it. Although the electrotactile sensation was not described as entirely natural, it was consistently rated as useful and, in most cases, pleasant. This corroborates the fact that even non-biomimetic stimuli can be meaningfully integrated into sensorimotor control when they are informative and well-encoded (Bach-y-Rita and Kercel, Reference Bach-y-Rita and Kercel2003). Notably, several participants anecdotally reported that the post-no-feedback condition felt more challenging, further supporting the notion that tactile feedback was beneficial even after short-term exposure. When asked to rank the different feedback modalities, participants expressed a clear preference for the spatial and mixed encoding schemes, which were almost equally favored. In contrast, frequency encoding was consistently ranked third, above the no-feedback condition but below the other two schemes, suggesting that while it provided some value, it was less effective or more cognitively demanding. The presence and embodiment were rated high even without feedback, which reflects the intrinsic immersion of VR, but mixed encoding still improved the embodiment, slightly but significantly.
Overall, the enhanced objective performance combined with favorable subjective evaluations highlights the practicality and robustness of the tested electrotactile feedback system – particularly when using combined encoding strategies – as a viable and effective solution for supporting sensorimotor control in immersive and real-world applications.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/wtc.2026.10049.
Data availability statement
The data that support the findings of this study are available from the corresponding author, S.D., upon request.
Author contribution
Conceptualization: E.V., M.A.G., C.C., H.K., S.D.; Data curation: E.V., M.A.G., S.D.; Data visualization: E.V.; Formal analysis: E.V., S.D.; Investigation: E.V.; Methodology: E.V., P.G.L., M.A.G., S.D.; Resources: M.S., E.H., S.D.; Software: E.V., P.G.L.; Supervision: M.A.G., H.K., C.C., S.D.; Validation: E.V., S.D.; Writing original draft: E.V., S.D.; Writing review and editing: All authors. All authors approved the final submitted draft.
Funding statement
This research was supported by grants from the European Union H2020 Program through the Project TACTILITY under Grant H2020-ICT-25-856718 and from INAIL (Istituto Nazionale per l’Assicurazione contro gli Infortuni sul Lavoro) under Grant Agreement PR23-PAS-P1 – DexterHand.
Competing interests
The author declares no competing interests exist.
Ethical standard
The research meets all ethical guidelines, including adherence to the legal requirements of the study country.
Open access
