• 2D Tactile Array Rendering and Information Cues (2003+)
• Revisited Vibrotactile Control Function (2009+)
• Force Feedback and Passive Haptics in Whole-hand Interaction (2005-2011)
• Force Feedback for Drawing on Data Surfaces (2010-2012)

2D Tactile Array (Tactile Grid) Rendering and Information Cues (2003+)

Research Assistants: Nicholas Lipari, Vijay Baiyya, Alp Asutay

This grid of tactors was created as a proof-of-concept for several rendering and cueing techniques. 2D tactile arrays may communicate position, direction, priority, etc. of objects in real or virtual environments. We developed a rendering framework including techniques such as spatial anti-aliasing for 2D tactile grids, level correction, and haptic glyphs, and we have conducted several experiments to evaluate them. These techniques allowed the rendering of smooth tactile motions (haptic strokes or paths) across the contact surface. More recently, we are investigating possible systematic distortions in user responses to array stimuli to improve understanding and calibrate for such distortions.

The array was demonstrated at IEEE Haptics 2004, Eurohaptics 2004, and IEEE VR. In demonstrations, it was driven by user input at a touchpad, allowing one user to haptically touch or “draw” on another user’s palm (related buzzwords: haptic interpersonal communication, mediated touch, remote social touch, tactile brush).

Demonstration of Position Accuracy Experiment

Publications

• Christoph W. Borst and Vijay B. Baiyya, “A 2D Haptic Glyph Method for Tactile Arrays: Design and Evaluation”, Worldhaptics 2009, pp. 599-604.
• Christoph W. Borst and Vijay B. Baiyya, “Enhancing VR-based Visualization with a 2D Vibrotactile Array”, ACM Virtual Reality Software and Technology (VRST) 2007, pp. 183-186.
• Christoph W. Borst and Alp V. Asutay, “Bi-level and Anti-aliased Rendering Methods for a Low-Resolution 2D Vibrotactile Array”, IEEE WorldHaptics 2005 conference, pp. 329-335. (PDF)
• Christoph W. Borst and Alp V. Asutay, “Bi-level and Anti-aliased Rendering Methods for a Low-Resolution 2D Vibrotactile Array”, IEEE WorldHaptics 2005 conference, pp. 329-335. (PDF)
• Christoph W. Borst and Charles D. Cavanaugh, “Touchpad-Driven Haptic Communication using a Palm-Sized Vibrotactile Array with an Open-Hardware Controller Design,” EuroHaptics 2004 conference.
• Hands-on demo:  IEEE VR 2009 (Palm-Sized Vibrotactile Array for VR-Based Visualization: Hardware Design, Communication Framework, and Evaluation).
• Hands-on demo: IEEE HAPTICS 2004 (vibrotactile array).
• Hands-on demo: EuroHaptics 2004 (touchpad-driven input and anti-aliased rendering for a vibrotactile array)
• Technical report: Christoph W. Borst and Charles D. Cavanaugh, “Haptic Controller Design and Palm-Sized Vibrotactile Array,” technical report. (link)
• Christoph W. Borst and Richard A. Volz, “Observations on and Modifications to the Rutgers Master to Support a Mixture of Passive Haptics and Active Force Feedback,” IEEE HAPTICS 2003, pp. 430-437. (link)

Revisited Vibrotactile Control Function (2009+)

Research Assistants: Nicholas Lipari

2D tactile arrays may be integrated into VR controllers, for example to enhance touch communication for collaborative tasks. Multiple tactors (tactile elements) may be activated in combination to approximate a vibration point (virtual tactor) having arbitrary position and intensity. We further studied the combination of vibration intensities from multiple tactors to guide virtual tactor rendering approaches. Multi-tactor vibrations were tested in 3-tactor and 4-tactor groups. Results suggest a relationship based on successively pairing physical tactors and virtual tactors to estimate total perceived loudness from individual tactor signal levels. The relationship leads us to propose a 2D bilog interpolation approach, provides a basis for assessment of existing rendering techniques, and lays a foundation for further study of 2D array rendering.

Four tactors labeled with intensities and interpolation parameters on a unit cell.

Vibrotactile control surface based on user data.

Publications

• Nicholas G. Lipari and Christoph W. Borst, “Study of 2D Vibration Summing for Improved Intensity Control in Vibrotactile Array Rendering”, ISVC (Visual Computing) 2014, pp. 325-334. (PDF)
• Nicholas G. Lipari and Christoph W. Borst, “Radial Expansion and an Effect of Visual Scale on the Spatial Perception of 2D Vibrotactile Position Cues”, IEEE VR 2009 Workshop on Perceptual Illusions in Virtual Environments
• Nicholas G. Lipari and Christoph W. Borst, “Contact Geometry and Visual Factors for Vibrotactile-Grid Location Cues”, ISVC (Visual Computing) 2010, pp. 729-738.
• Nicholas G. Lipari and Christoph W. Borst, “Investigation of Error in 2D Vibrotactile Position Cues with Respect to Visual and Haptic Display Properties: A Radial Expansion Model for Improved Cuing”, ICEIS International Conference on Enterprise Information Systems 2009, pp. 963-974.

Force Feedback and Passive Haptics in Whole-hand Interaction (2005-2011)

Research Assistants: Mores Prachyabrued, Arun Indugula

Whole-hand interactions in VR typically lack the motion constraints and touch sensation that would be present with real objects, leading to unrealistic and awkward results. We have studied the use of both static physical props and active hand-grounded force feedback to overcome such problems. This haptics work also contributed to haptic data compression research and to the development of haptic mixed reality.

Haptic Data Compression: At Worldhaptics 2005, we reported results of one of the first user-based evaluations of data compression in a force feedback loop, including logarithmic quantization that has both perceptual and statistical motivations (methods originally described in Borst’s 2002 dissertation and inspired by prior audio encoding techniques).

Haptic Mixed Reality: We also described the combination of cues from an active force feedback device and a static prop as a type of haptic mixed reality, and proposed the extension of Milgram’s taxonomy to multiple dimensions for mixed reality with both visual and haptic components.

Virtual Control Panel with physical prop and force glove (haptically-mixed reality)

Expansion of Reality-Virtuality Continuum to 2D, from Borst's thesis (2002)

Logarithmic encoder for data compression in force-feedback loop, Borst (2002)

Publications

• Mores Prachyabrued and Christoph W. Borst, “Dropping the Ball: Releasing a Virtual Grasp”, Proceedings of IEEE 3D User Interfaces (3DUI) 2011, pp. 59-66. (“runner-up best paper”). (PDF)
• Christoph W. Borst and Arun P. Indugula, “A Spring Model for Whole-hand Virtual Grasping”, Presence : Teleoperators and Virtual Environments, vol. 15 no. 1 (February 2006), pp. 47-61. (PDF)
• Christoph W. Borst and Arun P. Indugula, “Realistic Virtual Grasping,” IEEE Virtual Reality 2005 conference, pp. 91-98, 320. (awarded VR2005 honorable mention), (link)
• Christoph W. Borst, “Predictive Coding for Efficient Host-Device Communication in a Pneumatic Force-Feedback Display”, IEEE WorldHaptics 2005 conference, pp. 596-599. (link)
• Poster: Christoph W. Borst and Arun P. Indugula, “Whole-hand Grasping with Force Feedback,” IEEE WorldHaptics 2005.

Further information can be found at 3D Interactions.

Also see our video Realistic Virtual Grasping.

Force Feedback for Drawing on Data Surfaces (2010-2012)

Research Assistant: Phanidhar Raghupathy

We developed a data annotation interface for drawing directly on a data surface with a force-feedback stylus, allowing the user to feel the surface. We compute forces with a penalty-based method with force shading, optimized for a large terrain data structure: force magnitude is proportional to the stylus tip’s vertical distance below the surface, and direction is an interpolated surface normal. An experiment studied the effects of force feedback by comparing to a no-force condition and to a pseudohaptic condition. The pseudohaptic condition provided no real force but prevented the stylus from appearing to move below the surface with a constraining vertical offset to visual stylus position. Users traced paths at two difficulty levels (path along a trough or over moderately bumpy features).

Force feedback was found better than other conditions for the simpler path, but not better than pseudohaptics for the bumpy path. This adds to prior work that only considered trough-like features and comparison to no-force cases. We showed how the benefit of force hinges on the characteristics of the surface being marked, the benefit of visual constraints, and an additional subjective effect of the visual constraint (users reported increased sense of touch compared to the no-force case). The experiment also considered an interactive secondary view that allowed users to reach in and draw in a secondary view — a related project (3D Secondary Views: Windows, Lenses, and Mirrors) provides different types of secondary views.

Left: A user’s view of a terrain with (inset) a mirror-based 3D display and force stylus. The scene also shows a secondary view rectangle hovering above the terrain.
Right: Simple (top) and difficult (bottom) paths to trace in a force feedback experiment as described above.

Publications

• Poster: Phanidhar B. Raghupathy and Christoph W. Borst, “Force Feedback and Visual Constraint for Drawing on a Terrain: Path Type, View Complexity, and Pseudohaptic Effect”, 3DUI 2012, pp. 157-158. (PDF)
• Phanidhar Bezawada Raghupathy and Christoph W. Borst, “Investigation of Secondary Views in a Multimodal VR Environment: 3D Lenses, Windows, and Mirrors”, ISVC (Visual Computing) 2011, pp. 180-189. (PDF)