Exploring Virtual Worlds: Exploring Virtual Worlds Haptic and Tactile Feedback Russell Taylor March 1, 2000


What’s it all about?: What’s it all about? “Reach out and touch” the virtual world The only bi-directional modality Haptic Large-scale limb motions and forces: touching the world with a tool. Tactile Fingertip: texture, vibration, slippage The gray area: feeling denim with a pen


The Dream System (Holodeck): The Dream System (Holodeck) Large (essentially infinite) working volume Provides full-body contact at high resolution Combines haptic and tactile presentation As much force as required Holodeck safeties don’t reduce fidelity Incredibly complicated world model All of the above at once


The State of the Art: The State of the Art Small working volume (maybe 1 meter cube) At most three-fingered contact (usually 1 tool) Haptic or tactile presentation (mostly) Force can be easily overcome by user, or else the system is dangerous (trade off here) Maybe 10,000-polygon world model (Arthur Gregory and Ming Lin’s H-collide)


Haptic and Tactile Feedback: Haptic and Tactile Feedback What we will cover Example applications using haptic feedback Haptic perception (a tiny bit) Example haptic devices Haptic programming/system concerns Available local devices and libraries


Haptic and Tactile Feedback: Haptic and Tactile Feedback What we will not cover Details of tactile display See Welch notes for force lecture in ‘98 Physiology of haptic and kinesthetic perception I can get information if you are interested Force, friction, and adhesion models Chen et al. “Sticking to the Point” paper Seeger, Razzaque, Henderson work on multivariate perception


Example applications: Example applications Medical simulation Squeezing Flexible Objects Volume Sculpting Electronics training Drug/Protein docking simulation nanoManipulator


Applications: KISMET: Applications: KISMET The Karlsruhe Endoscopic Surgery Trainer Deforming objects: organs


Applications: Rutgers Master: Applications: Rutgers Master


Applications: GE Volume Sculpting: Applications: GE Volume Sculpting


Applications: MIT/RLE Training: Applications: MIT/RLE Training


Apps: Docker: Apps: Docker


Drug/Protein Docking: Drug/Protein Docking A “Lock and Key” Problem Natural use for force feedback Advantages: Up to a factor of 2 improvement in 6DOF positioning task Chemists said they understood more 6DOF in and out required


nanoManipulator: nanoManipulator


nanoManipulator characteristics: nanoManipulator characteristics Advantages: Finding the right point to modify/measure Control during modification “Haptic imaging”: more sensitive to facets, less damaging to the sample 6DOF in, 3DOF out Separate haptic and application threads


Haptic Perception (a tiny bit): Haptic Perception (a tiny bit)


Fingertip Parameters: Fingertip Parameters Spatial resolution of about 2.5 mm multiple forces closer are sensed as one Sensory bandwidth exceeds 500 Hz (also, system stability requires faster update for harder surfaces Fingers can comfortably apply forces at 5-10 Hz Impedence controller bandwidth of 1-2 Hz when squeezing an object


Slide18: Burdea & Coiffet, 1994


Example Devices: Example Devices


Immersion Corp: Impulse Engines (2D and 4D [pinch]): Immersion Corp: Impulse Engines (2D and 4D [pinch])


SensAble Technologies PHANToMs: SensAble Technologies PHANToMs


Cybernet Systems: 2D and 6D: Cybernet Systems: 2D and 6D


Sarcos Dextrous Master: Sarcos Dextrous Master


Haptic Programming, System Concerns: Haptic Programming, System Concerns


Concerns for Force Feedback: Concerns for Force Feedback Safety Update rates Force model


Force-Feedback Concerns: Safety: Force-Feedback Concerns: Safety It’s a robot; you’re in its working volume ARM: Foot-pedal cutoff (dead-man switch) PHANToM Premium: Safety glasses Damage to the device itself Thermal shutdown code for PHANToM Broken aluminum flying about


Concerns: High Update Rate: Concerns: High Update Rate Must be >500 Hz Required for stable hard surfaces Must be uninterrupted to prevent force discontinuities Much greater than graphics or simulation rates Graphics rate ~30-60Hz Simulation rate (as low as 1 Hz)


Solution: Dedicated Force Server: Solution: Dedicated Force Server Process dedicated to the force device Periodic position updates to application Presents local surface models (Adachi calls these intermediate representations) at high update rates (>500 Hz) Local approximations to the actual surface Must be rapid to compute


Intermediate representation: Plane: Intermediate representation: Plane Probe Local Plane Approximation (Used in Force Loop) Surface Complete Surface (known to Application)


Preventing discontinuity: Preventing discontinuity Discontinuity when plane updated Recovery time over several steps


Locally-Available Devices and How to Program Them: Locally-Available Devices and How to Program Them


Locally-available devices: Locally-available devices Several PHANToMs Program with VRPN Immersion/Logitech 2D joystick VRPN (You get to finish the driver) Argonne III Remote Manipulator (ARM) Program with ARMlib “Static Haptics” under the ceiling tracker Program using hand truck


Programming vrpn_ForceDevice: Programming vrpn_ForceDevice Open the device as a vrpn_Tracker Open the device again as a vrpn_Button Open the device again as a vrpn_ForceDevice Send it intermediate representations Plane, Point-to-point Spring, Mesh of Geometry Or, create your own (sword? Weigle doing volumes) Adjust their parameters Stiffness, friction, recovery time, location


Getting started with haptic programming: Getting started with haptic programming VRPN: Find someone who’s done it Russ Taylor, Adam Seeger, Amy Henderson Start with client_src/sphere_client.C GHOST You can run the demo apps on the machines Online manuals? Example programs?


References (From Greg Welch): References (From Greg Welch) Burdea, Grigore, and Philippe Coiffet, Virtual Reality Technology, 1994, John Wiley & Sons, ISBN 0-471-08632-0. Kalawsky, Roy S. The Science of Virtual Reality and Virtual Environments, 1993, Addison-Wesley, ISBN 0-201-63171-7. Cutkosky, M. and R. Howe, 1990, “Human Grasp Choice and Robotic Grasp Analysis,” Dextrous Robot Hands, S. Venkataraman and T. Iberall Eds., pp. 5-31, Springer Verlag. Shimoga, K., 1992, “Finger Force and Touch Feedback Issues in Dextrous Telemanipulation,” Proceedings of NASA-CIRSSE International Conference on Intelligent Robotic Systems for Space Exploration, Troy, NY, September. Stone, R., 1992, “Advanced Human-System Interfaces for Telerobotics Using Virtual Reality & Telepresence Technologies,” Proceedings of the Fifth International Conference on Advance Robotics (‘91 ICAR), Pisa, Italy, pp. 168-173


References (2): References (2) Stone, R., 1991, “Advanced Human-System Interfaces for Telerobotics Using Virtual Reality & Telepresence Technologies,” Proceedings of the Fifth International Conference on Advance Robotics (‘91 ICAR), Pisa, Italy, pp. 168-173 Stone, R., 1992, “Virtual Reality Tutorial,” MICAD Conference, Paris, France, pp. 200. Patrick, N., 1990, “Design, Construction, and Testing of a Fingertip Tactile Display for Interaction with Virtual and Remote Environments,” Masters Thesis, Department of Mechanical Engineering, MIT, August. Johnson, A., 1992, “Programmable Tactile Stimulator Array System and Method of Operation,” US Paten 5,165,897, November 24. Caldwell, D. and C. Gosney, 1993, “Enhanced Tactile Feedback (Tele-Taction) using a Multi-Functional Sensory System,” Proceedings of the 1993 International Conference on Robotics and Automation, Atlanta, GA, pp. 955-960, May.


References (3): References (3) Iwata, H., 1993, “Pen-based Haptic Virtual Environment,” Proceedings of IEEE Virtual Reality Annual International Symposium, Seattle, WA, pp. 287-292, September. Bouzit, M., P. Richard, and P. Coiffet, 1993, “LRP Dextrous Hand Master Control System,” Technical Report, Laboratoire de Robotique de Paris, 21 pp., January. Kramer, J., 1993, “Force Feedback and Texture Simulating Interfac Device,” US Patent 5,184,319, February 2. http://www.cs.unc.edu/Research/force http://www.cs.unc.edu/Research/vrpn