Haptics involves the modality of touch and the sensation of shape and texture an observer feels when virtually 'touching' an object. In our lab we have employed such devices as the PHANToM, a 6DOF force-feedback stylus, to enable users to explore the surfaces and textures of virtual objects and "palpate" deformable models of soft tissue in a medical simulator. We have also developed techniques for using a haptic stylus to "paint" on three-dimensional virtual objects, and for assisting visually impaired users to maintain orientation in virtual environments. We have created an architecture for what we refer to as "heterogeneous haptic collaboration," in which users with different types of haptic devices, such as a pen-based device like the PHANToM stylus and a glove-based device like the "CyberGrasp," an instrumented data glove with force-feedback exoskeleton, can touch each other's fingertips over the Internet and guide one another's exploration within a virtual environment. Most recently we have begun to develop strategies for more commonplace, inexpensive haptic devices, such as the Synaptics TouchPad, to enable us to predict user frustration with computer-based tasks from the user's patterns of application of force to the pressure-sensitive input device.
Our goal with respect to this particular line of research is to use the information from the pressure on the TouchPad to train a classifier to recognize negative user affect, and ultimately to dynamically alter task environment (difficulty level, help level, etc.) to adjust to the current state of the user. In the near future we expect to combine the haptic data with acoustic, lexical, discourse, and vision-based data streams for multimodal recognition of negative user affect. Currently our goals are to: (1) develop acquisition techniques for pressure data from the TouchPad that allow us to reliably describe normal patterns of positional and pressure data; (2) observe and log patterns of positional and pressure data under frustrating conditions; (3) from baseline and experimental conditions derive a model of positional and pattern differences which are predictive of negative user affective state and develop a function which reliably classifies new cases; (4) develop an adaptive user interface which adjusts task difficulty level and help dynamically when user frustration is recognized.