Συντάχθηκε 22-11-2011 13:38
από Eleni Stamataki
Email συντάκτη: estamataki<στο>tuc.gr
Ενημερώθηκε:
-
Ιδιότητα: σύνταξη/αποχώρηση υπάλληλος.
ΠΟΛΥΤΕΧΝΕΙΟ ΚΡΗΤΗΣ
Τμήμα Ηλεκτρονικών Μηχανικών και Μηχανικών Υπολογιστών
ΠΑΡΟΥΣΙΑΣΗ ΔΙΔΑΚΤΟΡΙΚΗΣ ΔΙΑΤΡΙΒΗΣ
“Blending Speech and Graphical User Interfaces
An empirical study on multimodal mobile interaction”
Περακάκης Εμμανουήλ
Δευτέρα 28 Νοεμβρίου 2011, Ώρα 13:00
Αμφιθέατρο κτ. Επιστημών, Πολυτεχνειούπολη
Εξεταστική Επιτροπή:
Αν. Καθ. Αλέξανδρος Ποταμιάνος, Τμήμα ΗΜΜΥ, Πολυτεχνείου Κρήτης (επιβλέπων)
Καθ. Διγαλάκης Βασίλης, Τμήμα ΗΜΜΥ, Πολυτεχνείου Κρήτης
Επ. Καθ. Αικατερίνη Μανιά, Τμήμα ΗΜΜΥ, Πολυτεχνείου Κρήτης
Καθ. Μιχάλης Ζερβάκης, Τμήμα ΗΜΜΥ, Πολυτεχνείου Κρήτης
Αν. Καθ. Ευριπίδης Πετράκης, Τμήμα ΗΜΜΥ, Πολυτεχνείου Κρήτης
Καθ. Ιωάννης Στυλιανού, Τμήμα Επιστήμης Υπολογιστών, Πανεπιστημίου Κρήτης
Ερευνητής Α, Γεράσιμος Ποταμιάνος, Ινστιτούτο Πληροφορικής & Τηλεπικοινωνιών, Δημόκριτος
ΠΕΡΙΛΗΨΗ
Mobile phones have already outnumbered personal computers. Although until recently the majority of phones were used mainly as voice communication devices, the emergence of powerful application-centric mobile devices such as personal digital assistants (PDAs) and smart-phones, has created excitement for the future of mobile computing. Despite the recent explosion of advanced mobile applications such as web browsing and video consuming, constraints such as reduced display size and limited input interaction methods pose new challenges for interaction designers. The use of more than one interaction modalities has been proposed as a possible solution to overcome these limitations. Multimodal interfaces process two or more combined user input modalities such as speech, pen or touch, in a coordinated manner with multimedia system output and can potentially offer more rich, robust and adaptive interaction experience.
This dissertation investigates multimodal interface design and evaluation with a focus on mobile interaction. One of the main aims is to showcase how to design information-filling multimodal systems that combine speech and graphical user interface (GUI) input (e.g. pen or touch). From the interaction design standpoint, the main focus is on identifying and exploiting the synergies resulting from the mixing of modalities in order to create robust and effective interfaces. The system designed and implemented, allows both unimodal and multimodal interaction and can be used across different platforms such as PCs, PDAs and mobiles phones such as the popular iPhone device.
For the evaluation of multimodal interaction both established and novel metrics are employed. Two new metrics were devised that measure the relation of input modality preferences to unimodal efficiency and the synergies found in a multimodal system. The proposed metrics, relative modality efficiency and multimodal synergy, can provide valuable information to the interaction design process of multimodal systems. Furthermore affective evaluation incorporating biosignals such as skin conductance and brain waves (EEG) has provided a rich amount of data not previously available. Use of such physiological channels and their elaborated interpretation is a challenging but also a potentially rewarding direction towards emotional and cognitive assessment of multimodal interface design.
Evaluation results show that multimodal systems can potentially outperform unimodal systems in terms of both performance and user satisfaction when designed to maximize the synergies between the modalities. Overall this research entails significant implications for designing efficient mobile interfaces.
