gw paper presentation

Discovering Gestures: 

Discovering Gestures A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI Michael Nielsen, Moritz Störring, Thomas Moeslund, Erik Granum Laboratory of Computer Vision and Media Technology Aalborg, Denmark

Motivation for new procedure: 

Motivation for new procedure

Human based approach: 

Human based approach A good gesture vocabulary may only match one specific application and user group Important principles Easy to perform Easy to remember Metaphorically and iconically intuitive Keep in mind Possible to recognise unambiguously

What do we need?: 

What do we need? Interdisciplinary Work Biomechanics Ergonomics Usability

Biomechanics and Ergonomics: 

Biomechanics and Ergonomics Constraints in postures and usage Inter- and intra finger relationships Wrist movement External and internal force Entire body posture

Usability in Gesture Interface: 

Usability in Gesture Interface Memorability/Efficiency Trade-off Keep number of gestures to a minimum Provide shortcut gestures to key functions Very simple dialogue Avoid spontaneous modifications Tolerance towards spontaneous moves

Procedure to find gesturing: 

Procedure to find gesturing Step 1. Find the Functions What are the users used to? What do they need? Step 2. Collect Gestures from User Domain Observation of gestures through scenarios and experiments Step 3. Extract Gesture Vocabulary Analyse video recordings from Step 2. Step 4. Benchmark Gesture Vocabulary Test gestures on the targeted users

Experiment – Step 1: Functions: 

Experiment – Step 1: Functions Architectural Design (Object manipulation) Proposed key functions Activate Menu Select Select All Insert Move Scale Rotate Delete Yes/Confirm No/Undo

Experiment – Step 2: User tests: 

Experiment – Step 2: User tests Video recorded sessions Scenarios The Pub Visit Noisy environment, verbal communication prevented. Spontaneous and forced gesturing HMC Static Cam

Experiment – Step 2: User tests: 

Experiment – Step 2: User tests HMC Static Cam Scenarios Planning of Furnishing Collaboration through verbal communication Spontaneous gesturing Simulated Furnishing Directing through gestures Conscious gesturing

Experiment – Step 2: User tests: 

Experiment – Step 2: User tests

Experiment – Step 3: Analysis: 

Experiment – Step 3: Analysis Observe in the video sequences Gesture variations Frequency for each gesture Dynamic or Static Evaluate gestures Ergonomics Functionality Recognition

Step 3: 

Step 3 Click to see

Video: 

Video

Experiment – Step 4: Benchmark: 

Experiment – Step 4: Benchmark Benchmark Semantics Memory Physical Stress Re-evaluate gestures Compare alternative gesture vocabularies Note: Memory mark is not normalized Compare user groups

Experiment – Step 4 - Results: 

Experiment – Step 4 - Results Fatigue Scoring: 1: No Problem 2: Mildly Tiring 3: Tiring 4: Annoying 5: Impossible

Conclusion: 

Conclusion Procedure usable Resulting gesture vocabulary Nice interpretation Easy to remember Mildly tiring to use gestures in general Benchmark tested on different users Give careful instructions to gestures More than just a posture and movement

Face and Gesture Network: 

Face and Gesture Network FG-Net EU IST funded program (2000-26434) Foresight reports Developing Corpora Catalyst to development Mailing list: http://www-prima.imag.fr/FGnet/html/home.html