Telehaptics
More actions
Telehaptics is the transmission of touch information (forces, motion, vibration, pressure and similar haptic signals) over a communication network, so that a person can feel a remote real environment, a shared virtual environment, or another person at a distance. It extends haptics from a local device to a networked link. A 2018 survey in IEEE Communications Surveys & Tutorials describes touch as the modality expected to complement audition and vision as a third media stream over the Internet.[1] Researchers also call the field haptic communication,[2] and since Gerhard Fettweis's 2014 paper much of the networking research has used the name Tactile Internet.[3]
Telehaptic systems range from force-reflecting robot teleoperation, where an operator's hand controller pushes back with the forces measured at a distant robot, to multi-user virtual worlds in which two people lift the same virtual object and feel each other's actions. For virtual reality and augmented reality the topic matters because touch places much tighter timing demands on a network than sight or hearing: kinesthetic feedback closes a control loop through the user's hand, and delay in that loop can make the interaction unstable as well as unconvincing.[4][2]
Definition and terminology
Telehaptic applications involve long-distance transfer of haptic, audio and visual information between users in different places, with separate quality of service requirements for each medium, according to a 2017 paper on telehaptic congestion control by Vineet Gokhale, Jayakrishnan Nair and Subhasis Chaudhuri of the Indian Institute of Technology Bombay.[5]
The haptic information itself is usually split into two submodalities. The IEEE 1918.1 working group describes tactile information as what is perceived by the mechanoreceptors of the skin, such as surface texture, friction and temperature, and kinesthetic information as what is perceived by the skeleton, muscles and tendons, such as force, torque, position and velocity.[4] The distinction has practical consequences. Kinesthetic exchange is closed loop: the operator's movement goes out and the resulting force comes back to the same hand. Tactile exchange is usually open loop, which relaxes its delay requirements.[4]
The IEEE 1918.1 working group defines the Tactile Internet as "A network (or network of networks) for remotely accessing, perceiving, manipulating, or controlling real or virtual objects or processes in perceived real time by humans or machines."[4] Telehaptics is one part of that broader concept, which also covers machine-to-machine control with no human in the loop. Human-in-the-loop interaction with haptic feedback is what the working group calls bilateral haptic teleoperation, and its stated goal is that users should not be able to tell a remote task from the same task done locally.[4]
How it works
Bilateral teleoperation
A typical telehaptic teleoperation system has a master device, operated by the user, and a slave device (the teleoperator) at the remote site. The two exchange haptic signals (forces, torques, position, velocity, vibration) together with video and audio over a network.[4] In a common position/force architecture the operator sends position and velocity to the remote robot, and the robot returns the interaction forces. Local control loops at each end are joined into a global control loop that is closed over the network, and kinesthetic signals are sampled at 1 kHz or more to keep that loop stable.[2]
Delay is the central problem. In 1965 William Ferrell reported that operators of a remote manipulator with transmission delay spontaneously adopted a "move-and-wait" strategy, making open-loop moves and then waiting to see the result.[6] Robert Anderson and Mark Spong showed in 1989, using passivity and scattering theory, why existing bilateral controllers became unstable for some environments under time delay, and proposed a control law that stays stable for any environment and any delay.[7] Günter Niemeyer and Jean-Jacques Slotine used wave variables in a passivity-based scheme for force-reflecting teleoperation with transmission delays, published in 1991.[8] Other stability-ensuring schemes for delayed kinesthetic communication include the time domain passivity approach and model-mediated teleoperation.[2]
Latency targets
Published latency targets differ by use case. The IEEE 1918.1 working group gives 1 to 10 ms for teleoperation in highly dynamic environments, 10 to 100 ms for medium-dynamic tasks such as telesurgery and telerehabilitation, and 100 ms to 1 s for static or quasi-static tasks such as telemaintenance. It notes that propagation delay alone limits a 1 ms round trip to end points about 150 km apart (about 100 km through fiber).[4] Gokhale and colleagues cite one-way delay and jitter limits of 30 ms and 10 ms for haptic data, against 150 ms and 30 ms for audio and 400 ms and 30 ms for interactive video.[5]
Haptic data reduction and codecs
Because kinesthetic samples are packetized and sent as soon as they are available, a 1 kHz signal produces a packet rate that is hard to sustain over a shared network such as the Internet. Block-based compression methods used for audio and video add processing delay and cannot be used here.[2] At the default rate of 1,000 packets per second, Gokhale and colleagues found that packet headers make up almost half of the telehaptic traffic.[5]
The main remedy is perceptual deadband coding. It uses Weber's law of just-noticeable differences: a new sample is sent only when it differs from the last transmitted value by more than a set fraction, and smaller changes are dropped as imperceptible. Steinbach and colleagues report that the just-noticeable difference for force lies between 7% and 15% in the literature, and that the deadband approach cuts the average packet rate by 80 to 90% while keeping a high quality of experience.[2] Tactile signals, which tolerate more delay, can use block-based processing and frequency-domain models of perception, much like audio coding. The IEEE codec group began its tactile work with vibrotactile signals.[4]
History
Remote manipulation predates networked haptics. Raymond Goertz of Argonne National Laboratory described a mechanical master-slave manipulator in the journal Nucleonics in November 1954.[9] Research on time-delayed manipulation in the 1960s, and on stable bilateral control in the late 1980s and early 1990s, laid the control foundations for later networked work.[6][7][8]
On 23 May 2002, the Touch Lab at the Massachusetts Institute of Technology and the Virtual Environments and Computer Graphics lab at University College London demonstrated what they called "transatlantic touch": a user at each site lifted a shared virtual box together through a networked virtual environment while feeling the other person's actions.[10][11] Both sites used the PHANToM haptic arm, which MIT's announcement said was invented at MIT in the early 1990s and sold commercially by SensAble Technologies. The announcement gave the travel time of the signal from MIT to London as 150 to 200 milliseconds depending on network traffic, and said each user had to perform the task very slowly or synchronization was lost.[11] MIT announced plans to demonstrate the system again at an Internet2 conference at the University of Southern California on 28-29 October 2002, and the team's 2004 paper states that it was presented at the Internet2 demonstration meeting that year. The team, which included Mel Slater and Mandayam Srinivasan, published a study of presence and copresence in the task in 2004.[10][11]
In Japan, Susumu Tachi's telexistence research added touch to remote robot avatars. The Japan Science and Technology Agency states that the TELESAR V robot, first developed in 2012, transmits sight, hearing and touch, using haptic transmission based on Tachi's "haptic primary colors" principle.[12] That principle treats force, vibration and temperature as the basic stimuli of skin contact. Tachi's laboratory describes pressure and vibration measured by the TELESAR V robot hand being presented through a display built into a glove, which applies pressure and shear force to the finger pad and vibration from its sides.[13]
Tactile Internet
According to the IEEE 1918.1 working group, the Tactile Internet concept originated at the Technical University of Dresden.[4] Gerhard Fettweis argued in IEEE Vehicular Technology Magazine that once networks reach a round-trip delay of about 1 ms, "human tactile to visual feedback control" becomes possible, so that wireless communications could carry control of real and virtual objects.[3] An ITU-T Technology Watch report on the Tactile Internet, published in August 2014 with Fettweis as first author, listed robotics and telepresence, virtual reality and augmented reality among its application fields and stated that 1 ms end-to-end latency is necessary for Tactile Internet applications.[14] For virtual reality, the report described "Shared Haptic Virtual Environments" in which several users are physically coupled through a VR simulation, and noted that physics simulation and haptic display typically update at about 1,000 Hz, which corresponds to an ideal round-trip latency of 1 ms.[14]
Standards
The IEEE 1918.1 working group grew out of a proposal by King's College London and the Technical University of Dresden to an IEEE Communications Society meeting in Santa Clara, California, in November 2015; its project authorization request was approved in March 2016.[4] The working group's use cases include teleoperation, immersive virtual reality, interpersonal communication and live haptic-enabled broadcast. For immersive VR it cites a motion-to-photon target below 20 ms and concludes that the network delay for haptic feedback should stay below about 10 ms.[4] Its paper also states that the Tactile Internet is meant to run as an overlay on 5G or any other network that meets a use case's end-to-end requirements, not on 5G alone.[4]
| Standard | Subject | Status |
|---|---|---|
| IEEE 1918.1-2024 | "IEEE Standard for Tactile Internet--Application Scenarios, Definitions and Terminology, Architecture, Functions, and Technical Assumptions": a framework and reference architecture covering mission-critical uses (manufacturing, transportation, healthcare, mobility) and noncritical ones (edutainment, events)[15] | Active; board approval 26 September 2024, published 23 January 2026[15] |
| IEEE 1918.1.1-2024 | "IEEE Standard for Haptic Codecs for the Tactile Internet": a no-delay kinesthetic codec, a delay-robust kinesthetic codec, a tactile codec, and handshaking and metadata exchange for haptic device capabilities[16] | Active; board approval 15 February 2024, published 14 June 2024[16] |
| ISO/IEC 23090-31:2025 | MPEG-I haptics coding: coded representation of vibrotactile and kinaesthetic haptic media, in a human-readable JSON format and a compressed bitstream, for playback of immersive experiences[17] | First edition, January 2025[17] |
The call for participation in the haptic codec effort scheduled its kick-off meeting for 8-9 December 2016 in Fairfax, Virginia, near the IEEE GLOBECOM 2016 conference. It invited manufacturers of haptic input and output devices, providers of remote applications such as tele-surgery, tele-training, tele-rehabilitation and tele-robotics, and the gaming industry.[18] IEEE lists an open-source implementation under the BSD 3-Clause license.[16]
Applications in VR and AR
The 2002 MIT and UCL experiment was an early test of haptic collaboration in a shared virtual environment over a long-distance network, and its authors described techniques for reducing the instability that network latency caused in the haptic interaction.[10] The ITU report later presented shared haptic virtual environments as a main VR use of the Tactile Internet, with tele-surgery and micro-assembly training as examples of tasks that need fine force feedback between users.[14] Steinbach and colleagues list touch-augmented VR systems and telepresence calls with touch among the applications that motivate haptic codecs.[2]
Robotic avatars
Robotic avatar systems combine a VR headset for vision with telehaptic feedback for the hands. Team NimbRo of the University of Bonn won the US$5 million grand prize of the ANA Avatar XPRIZE at the finals in Long Beach, California, on 5 November 2022.[19][20] According to the team's paper, the operator wore a Valve Index head-mounted display, arm exoskeletons with force/torque feedback, and SenseGlove DK1 hand exoskeletons that gave force and haptic feedback to the fingertips. Contact sensors in the robot's fingertips drove that feedback.[21] The final task required the operator to pick a rough-textured stone by touch. For it, microphones in one robot fingertip picked up vibrations, a neural network classified short segments as rough or smooth, and a vibration actuator on the operator's fingertip displayed the result.[21]
Social touch
Mediated social touch, such as a handshake, pat or hug conveyed between remote people, is one of the IEEE 1918.1 use cases under the name haptic interpersonal communication. The working group notes that the remote participant can be represented by a social robot or by a virtual reality avatar.[4] Gijs Huisman's 2017 survey in IEEE Transactions on Haptics reported that research on technology-mediated social touch has found effects similar to those of actual social touch, and stressed the role of stimulus qualities, multimodal cues and context.[22] A 2024 systematic review in Virtual Reality by Giulio Jacucci and colleagues covered a decade of research on touch between avatars, and between people and virtual agents, in VR. It called for more in-the-wild studies and attention to emerging issues in social VR.[23]
Research
Open research problems include combining data reduction with stability-preserving control. Steinbach and colleagues note that the original wave-variable, passivity and model-mediated schemes ignored the high kinesthetic packet rate. They report that deadband coding combined with wave variables suits only constant delay, while a later combination with the time domain passivity approach remains stable under time-varying and unknown delays.[2] Other work targets network transport: Gokhale and colleagues proposed a transport-layer protocol that merges haptic samples into fewer packets when a shared network is congested, and tested it with a real-time telepottery experiment.[5] On the network side, the IEEE 1918.1 working group envisions network support functions that could model the remote environment, for example with artificial intelligence, so that telehaptic sessions can span distances greater than a 1 ms propagation budget allows.[4]
See also
References
- ↑ Konstantinos Antonakoglou, Xiao Xu, Eckehard Steinbach, Toktam Mahmoodi, Mischa Dohler (2018). "Toward Haptic Communications Over the 5G Tactile Internet". IEEE Communications Surveys and Tutorials, vol. 20, no. 4, pp. 3034-3059. https://doi.org/10.1109/COMST.2018.2851452. Retrieved 2026-10-06.
- ↑ 2.0 2.1 2.2 2.3 2.4 2.5 2.6 2.7 Eckehard Steinbach, Matti Strese, Mohamad Eid, Xun Liu, Amit Bhardwaj, Qian Liu, Mohammad Al-Ja'afreh, Toktam Mahmoodi, Rania Hassen, Abdulmotaleb El Saddik, Oliver Holland (February 2019). "Haptic Codecs for the Tactile Internet". Proceedings of the IEEE, vol. 107, no. 2, pp. 447-470. https://doi.org/10.1109/JPROC.2018.2867835. Retrieved 2026-10-06.
- ↑ 3.0 3.1 Gerhard P. Fettweis (March 2014). "The Tactile Internet: Applications and Challenges". IEEE Vehicular Technology Magazine, vol. 9, no. 1, pp. 64-70. https://doi.org/10.1109/MVT.2013.2295069. Retrieved 2026-10-06.
- ↑ 4.00 4.01 4.02 4.03 4.04 4.05 4.06 4.07 4.08 4.09 4.10 4.11 4.12 4.13 Oliver Holland, Eckehard Steinbach, R. Venkatesha Prasad, et al. (February 2019). "The IEEE 1918.1 "Tactile Internet" Standards Working Group and its Standards". Proceedings of the IEEE, vol. 107, no. 2, pp. 256-279. https://doi.org/10.1109/JPROC.2018.2885541. Retrieved 2026-10-06.
- ↑ 5.0 5.1 5.2 5.3 Vineet Gokhale, Jayakrishnan Nair, Subhasis Chaudhuri (2017-01-12). "Congestion Control for Network-Aware Telehaptic Communication". arXiv (cs.NI), 1610.00609v2. https://arxiv.org/abs/1610.00609. Retrieved 2026-10-06.
- ↑ 6.0 6.1 William R. Ferrell (September 1965). "Remote manipulation with transmission delay". IEEE Transactions on Human Factors in Electronics, vol. HFE-6, no. 1, pp. 24-32. https://doi.org/10.1109/THFE.1965.6591253. Retrieved 2026-10-06.
- ↑ 7.0 7.1 R. J. Anderson, M. W. Spong (May 1989). "Bilateral control of teleoperators with time delay". IEEE Transactions on Automatic Control, vol. 34, no. 5, pp. 494-501. https://doi.org/10.1109/9.24201. Retrieved 2026-10-06.
- ↑ 8.0 8.1 G. Niemeyer, J.-J. E. Slotine (January 1991). "Stable adaptive teleoperation". IEEE Journal of Oceanic Engineering, vol. 16, no. 1, pp. 152-162. https://doi.org/10.1109/48.64895. Retrieved 2026-10-06.
- ↑ R. C. Goertz (November 1954). "Mechanical master-slave manipulator". Nucleonics, vol. 12, no. 11 (OSTI record 4399868). U.S. Department of Energy Office of Scientific and Technical Information. https://www.osti.gov/biblio/4399868. Retrieved 2026-10-06.
- ↑ 10.0 10.1 10.2 Jung Kim, Hyun Kim, Boon K. Tay, Manivannan Muniyandi, Mandayam A. Srinivasan, Joel Jordan, Jesper Mortensen, Manuel Oliveira, Mel Slater (June 2004). "Transatlantic Touch: A Study of Haptic Collaboration over Long Distance". Presence: Teleoperators and Virtual Environments, vol. 13, no. 3, pp. 328-337. https://doi.org/10.1162/1054746041422370. Retrieved 2026-10-06.
- ↑ 11.0 11.1 11.2 "MIT and London team report first transatlantic touch". MIT News. Massachusetts Institute of Technology. 2002-10-28. https://news.mit.edu/2002/touchlab3. Retrieved 2026-10-06.
- ↑ "Realization of an alter-ego robot capable of transmitting haptic sensations". Japan Science and Technology Agency. 2016. https://www.jst.go.jp/EN/achievements/research/susumu_tachi2016.html. Retrieved 2026-10-06.
- ↑ "Haptic Primary Colors". Tachi Laboratory. The University of Tokyo. https://tachilab.org/en/about/hpc.html. Retrieved 2026-10-06.
- ↑ 14.0 14.1 14.2 G. Fettweis, H. Boche, T. Wiegand, et al. (August 2014). "The Tactile Internet". ITU-T Technology Watch Report. International Telecommunication Union. https://www.kom.tu-darmstadt.de/papers/FBW+14-1.pdf. Retrieved 2026-10-06.
- ↑ 15.0 15.1 "IEEE 1918.1-2024: IEEE Standard for Tactile Internet--Application Scenarios, Definitions and Terminology, Architecture, Functions, and Technical Assumptions". IEEE Standards Association. https://standards.ieee.org/ieee/1918.1/6721/. Retrieved 2026-10-06.
- ↑ 16.0 16.1 16.2 "IEEE 1918.1.1-2024: IEEE Standard for Haptic Codecs for the Tactile Internet". IEEE Standards Association. https://standards.ieee.org/ieee/1918.1.1/6835/. Retrieved 2026-10-06.
- ↑ 17.0 17.1 "ISO/IEC 23090-31:2025, Information technology, Coded representation of immersive media, Part 31: Haptics coding (sample pages)". iTeh Standards. ISO/IEC. January 2025. https://cdn.standards.iteh.ai/samples/86122/9c7cbf14909b4a429f5986792a4128e7/ISO-IEC-23090-31-2025.pdf. Retrieved 2026-10-06.
- ↑ "Call for Participation: IEEE Haptic Codecs for the Tactile Internet". IEEE 1918.1 Tactile Internet Working Group. IEEE Standards Association. 2016. https://grouper.ieee.org/groups/1918/1/haptic_codecs/files/Call_for_Participation_P1918.1.1.pdf. Retrieved 2026-10-06.
- ↑ "NimbRo announced as winner of the $10M ANA Avatar XPRIZE". XPRIZE. 2022-11-05. https://www.xprize.org/news/ana-avatar-xprize-winners. Retrieved 2026-10-06.
- ↑ Ian Hamilton (2022-11-07). "NimbRo Wins $5 Million Avatar XPRIZE Driving Robot With VR Headset". UploadVR. https://www.uploadvr.com/nimbro-wins-5-million-xprize/. Retrieved 2026-10-06.
- ↑ 21.0 21.1 Max Schwarz, Christian Lenz, Raphael Memmesheimer, Bastian Pätzold, Andre Rochow, Michael Schreiber, Sven Behnke (2023-12-06). "Robust Immersive Telepresence and Mobile Telemanipulation: NimbRo wins ANA Avatar XPRIZE Finals". IEEE-RAS International Conference on Humanoid Robots (Humanoids) 2023, arXiv 2303.03297v3. https://arxiv.org/abs/2303.03297. Retrieved 2026-10-06.
- ↑ Gijs Huisman (2017). "Social Touch Technology: A Survey of Haptic Technology for Social Touch". IEEE Transactions on Haptics, vol. 10, no. 3, pp. 391-408. https://doi.org/10.1109/TOH.2017.2650221. Retrieved 2026-10-06.
- ↑ Giulio Jacucci, Andrea Bellucci, Imtiaj Ahmed, Ville Harjunen, Michiel Spape, Niklas Ravaja (2024-11-14). "Haptics in social interaction with agents and avatars in virtual reality: a systematic review". Virtual Reality, vol. 28, no. 4. https://doi.org/10.1007/s10055-024-01060-6. Retrieved 2026-10-06.