LLDN Health FCL Wallscreen
From AMI@Work Communities Wiki
Wallscreen, touch and feel user interface
The lab provides an intelligent care infrastructure, consisting of different mobile and integrated devices, for supporting elderly and handicapped people. The setup of the lab enables in-situ evaluations of new care concepts and medical technologies by observing different target user populations in realistic usage situations. As the lab relies on a modular technical concept, it can be expanded with other technical products, systems and functionalities, in order to address different user groups as well as individuals with differences in their cognitive, health-related or cultural needs. The lab is equipped with an interactive wall, which was used as an output channel for all tests. The wall consists of six rear projection display elements and measures 4.8m x 2.4m with a total resolution of 3072 x 1536 pixels.
Input devices
Direct Multi-Touch Control. Multi-touch input is realized by Rear Diffuse Illumination technique. The display elements are illuminated with infrared light from behind. The back of the acrylic wall surface diffusely reflects the infrared light and thereby avoids specular highlights, which can result from the infrared illumination. Infrared light reflected from fingers touching the wall is captured by six PointGrey FireflyMV cameras. The Community Core Vision software is processing the resulting bright blobs in the captured images. Six instances of this software, one for each camera, send Tangible User Interface Object (TUIO) messages to the demo application. These TUIO messages contain the touch coordinates and unique touch IDs. The demo application maps the touches to a common coordinate system and ensures that their IDs remain unique.
Remote Trackpad Control. For realizing remote trackpad control, we use the touch screen of an Apple iPod touch as a trackpad. The iPod touch is intentionally called trackpad, since the integrated display is not used. The users’ locus of attention should reside on the wall-sided display. When the device is touched, events containing the touch coordinates are sent to the demo application. The events are processed to generate relative input. The common trackpad gestures to trigger a click or start a drag-and-drop operation are used: Users can singletap the display to click, or single-tap once and then touch the surface again to start a drag operation.
Remote Gesture Control. We use reflective infrared markers, which are attached to the users’ hand to create a robust hand tracking system. One marker is attached to the back of the user’s hand. The position of this marker defines the pointing position. A second marker is attached to the user’s forefinger, which allows the detection of clicking gestures. Reflected infrared signals are captured with a single commercial tracking camera (Optitrack FLEX: V100 from NaturalPoint) located behind the users shoulder. The hand tracking system was designed to be used by a sitting, non-moving user. During the system evaluation an individual coordinate frame, which depends for example on the user’s sitting position and arm length, was defined for each user. Technically, the marker tracking was executed inside the driver from NaturalPoint, which runs on a Windows XP computer. The position and the hand state (pointing or clicking) were sent over the network to the demo application.
